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Abstract 

The  strategic  use  of  Information  Technology  in  the  acquisition  field  can  be  very 
usefiil  in  the  decision-making  process  of  evaluating  alternative  solutions  during  a 
Government  source  selection.  Current  implementation  of  information  technology  provides 
a  more  tactical  approach  to  systems  development.  The  use  of  Electronic 
Commerce/Electronic  Data  Interchange  and  the  internet  to  electronically  transfer 
information  is  only  the  beginning  of  the  shift  towards  a  more  strategic  design  process  for 
information  systems  within  Government  procurement  agencies.  A  schematic  model  was 
designed  to  demonstrate  how  information  technology,  such  as  Decision  Support  Systems, 
Expert  Systems,  and  Shared  Data  Warehousing  could  assist  the  SSA  in  selecting  the 
optimal,  or  best  value  solution.  In  addition,  three  source  selection  evaluation  models 
using  management  science  techniques  were  designed  and  developed  using  Microsoft  Excel 
software.  The  Sealed  Bidding,  FAR  Part  14,  and  Competitive  Proposal,  FAR  Part  15 
models  implemented  Integer  Linear  Programming  through  Microsoft  Excel’s  SOLVER 
option.  The  AFFARS  Appendix  AA/BB  model  implemented  the  use  of  the  multi-criteria 
Analytical  Hierarchy  Process. 
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AN  EXPLORATORY  STUDY  ON  THE  STRATEGIC  USE  OF 
INFORMATION  TECHNOLOGY  IN  THE  SOURCE  SELECTION 
DECISION-MAKING  PROCESS 


1.  Introduction 


1.1  Background 

Acquisition  Reform  mandates  have  led  Department  of  Defense  (DoD)  policy 
makers  and  acquisition  personnel  to  continually  search  for  new  initiatives  to  improve  the 
acquisition  process.  Reducing  acquisition  lead-time  is  a  major  component  to  ensure 
quality  products  are  delivered  to  the  right  place,  in  the  right  quantity,  and  at  the  right  time 
(Dobler  and  Burt,  1996:  42).  As  the  DoD  budget  continues  to  shrink,  acquisition 
personnel  must  initiate  strategic  measures  to  maximize  the  benefits  received  from  every 
tax  dollar  spent.  Policies  which  implement  philosophies  such  as  using  Cost  as  an 
Independent  Variable  (CATV)  to  assist  with  reducing  costs;  using  “Past  Performance”  as 
an  evaluation  criterion  to  hold  a  contractor  accountable  for  his/her  poor  past  performance 
to  reduce  the  Government’s  risk;  and  using  a  Statement  of  Objectives  rather  than  a 
Statement  of  Work  to  shift  design  and  development  risk  from  the  Air  Force  to  the 
contractor  are  just  some  examples  in  which  the  DoD  and  the  Air  Force  are  changing  their 
strategies  in  the  acquisition  field.  These  strategies,  and  others  like  them,  have  increased 
the  complexity  for  decision-makers  in  their  attempt  to  select  the  best  source  of  supply  in 
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many  types  of  acquisitions.  In  his  book,  Decision  Making  in  the  Purchasing  Process, 
Phillip  White  states,  “we  (as  purchasing  managers)  must  develop  realistic  and  flexible 
guidelines  that  recognize  the  (evaluation)  factors  involved  and  deal  with  them  in  an 
informed  way”  (White,  1978:  9).  The  increasing  complexities  from  these  ever-changing 
strategies  augment  the  need  for  a  more  systematic  approach  to  assist  in  the  decision¬ 
making  process  during  source  selections.  The  purpose  of  this  research  is  to  explore 
various  strategically-focused  information  technology  tools  which  might  be  utilized  in  the 
source  selection  process  in  an  attempt  to  move  the  acquisition  field  from  a  reactive  or 
tactical  focus  to  a  more  proactive  or  strategic  focus.  The  flow  of  information  required  by 
personnel  in  the  source  selection  decision-making  evaluation  process  is  analyzed  to  see 
how  information  technology  can  be  incorporated  to  provide  more  reliable  and  accessible 
data.  The  evaluation  process  is  then  supported  using  a  commercially  available  software 
tool  for  solving  “optimization”  problems.  This  software  tool  is  examined  to  determine  the 
extent  it  can  assist  decision-making  in  the  source  selection  process. 

1.2  General  Issue 

H.  B.  Twyford  of  the  Otis  Elevator  Company,  in  one  of  the  earliest  books  focused 
on  purchasing,  wrote: 

A  (purchasing)  staff  which  is  entirely  unsympathetic  with  the  particular 
needs  of  the  users  of  the  material  will  fail  to  grasp  what  is  one  of  the  most 
essential  things  for  their  department.  They  will  be  dealing  with  papers  and 
accounts  instead  of  with  men  and  things.  (Dobler  and  Burt,  1996:  7) 

According  to  Dobler  and  Burt,  companies  in  the  commercial  market  have 

historically  focused  their  profit-gaining  strategies  toward  efforts  to  improve  marketing. 
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R&D,  finance,  and  operation  functions.  Until  approximately  the  1970s,  commercial 
purchasing  personnel  possessed  neither  the  skills  nor  the  aptitude  to  assist  the  purchasing 
function  achieve  its  fullest  contribution  towards  the  success  of  the  organization.  The 
purchasing  department  was  responsible  for  a  significant  portion  of  the  costs  of  goods  sold, 
and  purchased  materials  were  the  cause  of  a  large  portion  of  a  companies  quality 
problems.  “By  the  late  1980s,  material  costs  made  up  approximately  60  percent  of  the 
cost  of  goods  sold  in  the  United  States”  (Dobler  and  Burt,  1996:  3-38).  As  a  result,  the 
impact  of  purchasing  and  materials  management  on  a  company’s  assets  became  extremely 
significant  and  very  visible  (Dobler  and  Burt,  1996:  3-38).  To  remain  competitive, 
companies  began  to  look  at  the  purchasing  process  as  an  area  to  cut  costs.  “Two  major 
paradigm  shifts  occurred:  (1)  a  shift  from  a  focus  on  internal  processes  to  value-added 
benefits  and  (2)  from  a  tactical  to  a  strategic  focus”  (Dobler  and  Burt,  1996:  9). 

Successful  companies  began  to  view  their  supply  or  acquisition  strategies  as  a  strategic 
weapon  as  important  as  their  marketing,  conversion  and  financial  strategies  (Dobler  and 
Burt,  1996:  9).  “The  extent  to  which  the  purchasing  function  will  continue  to  contribute 
to  the  achievement  of  organizational  goals  depends  on  the  ability  of  purchasing  managers 
to  question  the  conventional  ways  of  doing  things  and  to  keep  up  with,  and  actively 
develop,  new  ideas.  Continued  systematic  review  of  the  variables  important  in  purchasing 
decisions  provides  an  essential  tool  to  accomplish  this  objective”  (White,  1978:  6). 

Similar  to  the  commercial  market’s  concern  to  increase  profit,  the  DoD  is 
concerned  with  its  ability  to  control  and  reduce  costs.  The  DoD  is  similar  to  the 
commercial  market  in  its  attempt  to  acquire  quality  products  while  controlling  costs,  with 
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one  major  difference:  its  strategic  focus  on  decision-making.  This  exception  is  clearly 
evident  in  the  design  and  implementation  of  information  technology  systems  within  the 
acquisition  arena.  In  a  23  Oct  97  Background  Paper  on  Revolution  in  Business  Affairs 
(RBA),  SAF/AQXA  discusses  Dr.  Hamre’s  (DepSecDef)  seventeen  “Management  Reform 
Memorandums”  designed  to  initiate  a  “revolution  in  business  affairs”  across  the 
Department  of  Defense.  These  memos  were  written  in  response  to  Secretary  Cohen’s  call 
for  infrastructure  reductions  in  the  Quadrennial  Defense  Review.  Management  Reform 
Memorandum  #2  requires  the  movement  to  a  Paper  Free  Contracting  Process  by  1  Jan 
2000  (SAF/AQXA,  1997).  These  memorandums  were  the  result  of  the  implementation  of 
Section  30  of  the  Office  of  Federal  Procurement  Policy  (OFPP)  Act  (41  U.S.C.  426) 
which  requires  the  development  and  implementation  of  an  automated  system  to  create  a 
paperless  environment. 

Review  of  the  Federal  Acquisition  Regulations  (FAR),  Subpart  4.504  discusses  the 
functions  of  the  Federal  Acquisition  Computer  Network  (FACNET),  which  according  to 
FAR  4.500  is  the  current  Government  Electronic  Commerce/Electronic  Data  Interchange 
(EC/EDI)  system  implemented  as  required  by  Section  30  of  the  OFPP  Act.  According  to 
the  FAR,  the  purpose  of  the  automated  system  is  to  transmit  and  store  information 
electronically.  FAR  4.504  states  FACNET’s  functions: 

(a)  For  agencies— 

(1)  Provide  widespread  public  notice  of  contracting  opportunities,  and 
issue  solicitations; 

(2)  Receive  responses  to  solicitations  and  associated  requests  for 
information; 

(3)  Provide  widespread  public  notice  of  contract  awards  and  issuance  of 
orders  (including  price); 

(4)  Receive  questions  regarding  solicitations,  if  practicable; 
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(5)  Issue  contracts  and  orders,  if  practicable; 

(6)  Initiate  payments  to  contractors,  if  practicable;  and 

(7)  Archive  data  relating  to  each  procurement  action. 

(b)  For  the  private  sector— 

(1)  Access  notices  of  solicitation; 

(2)  Access  and  review  solicitations; 

(3)  Respond  to  solicitations; 

(4)  Receive  contracts  and  orders,  if  practicable; 

(5)  Access  information  on  contract  awards  and  issuance  of  orders;  and 

(6)  Receive  payment  by  purchase  card,  electronic  funds  transfer,  or 

other  automated  means,  if  practicable. 

On  12  July  96,  Ms.  Eleanor  R.  Spector,  Director  of  Defense  Procurement,  Office 
of  the  Under  Secretary  of  Defense  signed  a  letter  to  the  Directors  of  Defense  Agencies 
mandating  the  use  of  a  Standard  Procurement  System  (SPS)  as  the  standardized 
automated  procurement  system  for  use  by  the  DoD  procurement  offices.  According  to 
the  Navy,  SPS  is  “the  next  generation  of  procurement  application  software,  designed  to 
link  acquisition  reform  and  common  DOD  procurement  business  processes  with 
commercial  best  practices  and  advances  in  electronic  commerce”  (DoD  SPS,  1998). 

SPS  is  an  integrated  system  consisting  of :  (DoD  SPS,  1998) 

(1)  Licensed  access  to  the  contractor-furnished  SPS  application  software; 

(2)  Government  furnished  data; 

(3)  A  relational  data  base  management  system; 

(4)  Installed  government  operating  environment; 

(5)  System  connectivity  (e.g.,  local  area  or  wide  area  network); 

(6)  A  Shared  Data  Warehouse  (SDW). 

The  SPS  is  currently  being  designed  to  support  DOD  procurement  functions  that 
acquire  systems,  supplies  and  services.  The  vision  is  for  data  input  to  begin  with  receipt  of 
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a  requirement  and  to  finish  at  contract  close-out.  The  standard,  automated  procurement 
functions  performed  during  this  process  include:  (DoD  SPS,  1998) 

(1)  Collecting  the  requirements; 

(2)  Determining  the  appropriate  method  for  acquiring  systems,  supplies  or 

services; 

(3)  Soliciting  and  selecting  sources; 

(4)  Awarding,  reporting,  modifying,  terminating  and  closing  out  contracts  and 

other  instruments; 

(5)  Inspecting  and  accepting  systems,  supplies  or  services; 

(6)  Monitoring  and  administering  quality  assurance  actions  and  programs; 

(7)  Production  and  engineering  surveillance; 

(8)  Property  administration; 

(9)  Determining  amounts  payable; 

(10)  Monitoring,  approving  and  tracking  payments. 

While  the  current  use  of  these  EC/EDI  technologies  in  the  acquisition  field  are 
broad  in  scope,  attempting  to  standardize  and  unify  Government  acquisition  offices,  the 
effort  is  still  limited  in  application.  An  information  system  is  defined  as  “a  set  of 
interrelated  components  that  collect  (or  retrieve),  process,  store,  and  distribute 
information  to  support  decision-making  and  control  in  an  organization”  (Laudon  and 
Laudon,  1995:  6).  In  addition,  information  systems  assist  managers  and  employees  to 
analyze  problems,  visualize  complex  subjects,  and  create  new  products  (Laudon  and 
Laudon,  1995).  According  to  FAR  2. 101,  information  technology  is  defined  as  “any 
equipment,  or  interconnected  system(s)  or  system(s)  of  equipment,  that  is  used  in  the 
automatic  acquisition,  storage,  manipulation,  management,  movement,  control,  display, 
switching,  interchange,  transmission,  or  reception  of  data  or  information  by  the  agency” 
(FAR  2. 101). 

Using  an  analogy  to  H.  B.  Twyford’s  quote  on  purchasing.  Government  personnel 
responsible  for  the  implementation  of  information  technology  in  the  contracting/ 
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acquisition  field  must  remember  to  “deal  with  men  and  things  instead  of  papers  and 
accounts”  (Dobler  and  Burt,  1996:  7).  The  current  SPS  design  (version  4.0)  has  a  limited 
ability  to  assist  the  SSA  with  the  manipulation  and  analysis  of  evaluation  information  for 
decision-making.  In  offer  evaluation,  the  system  allows  the  contract  specialist  to  evaluate 
quote  information  from  all  vendors,  simultaneously,  by  line  item  and  provides  a  method 
for  totaling  calculations.  In  addition,  SPS  can  generate  and  analyze  price  history  for  a 
product  by  conducting  price  searches  within  the  database  to  locate  previous  acquisitions 
of  the  same  product  (DoD  SPS,  1998).  With  proper  planning  and  design,  information 
technology  can  be  used  to  strategically  take  the  process  further  into  analyzing  and 
manipulation  of  the  information  collected,  compiled,  and  stored.  The  creation  of  an 
automated  information  system,  such  as  FACNET,  for  the  transmittal  and  receipt  of 
information,  and  storage  of  these  transactions  demonstrates  a  tactical  focus  of  information 
technology  in  the  contracting  field.  A  paradigm  shift  in  the  design  and  development  of 
information  technology  systems  to  a  more  strategic  focus  is  required  (Dobler  and  Burt, 
1996:  7).  The  SPS  design  and  development  is  only  the  beginning  of  the  shift  from  a 
tactical  to  strategic  approach  to  decision-making  in  the  acquisition  field.  Information 
technology  can  process  and  manipulate  data  to  assist  with  decision-making.  Numerous 
software  tools  can  choose  an  optimal  solution  from  among  different  alternates  using  multi¬ 
criteria  evaluation  factors,  perform  risk  assessments,  and  forecast  the  probability  of  an 
event  occurring,  such  as  the  likelihood  of  a  project  failure  if  a  contract  is  awarded  to  a 
specific  contractor.  Implementation  and  use  of  these  tools  in  a  systematic  method  can 
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help  shift  Government  purchasing  from  a  tactical,  reactive  mode  of  operation  to  a  more 
strategic,  proactive  mode  of  operation. 

1.3  Specific  Problem 

The  specific  focus  of  this  research  is  to  survey  current  information  technology 
components  to  find  tools  which  can  strategically  assist,  either  directly  or  indirectly,  the 
Source  Selection  Authority  (SSA)  with  the  evaluation  and  analysis  of  alternative  sources 
in  the  decision-making  process  of  a  competitive  source  selection.  Only  components  which 
can  enhance  the  flow  of  information  into  the  source  selection  evaluation  process  to  assist 
with  the  manipulation  or  processing  of  required  evaluation  data  will  be  identified.  Various 
software  programs  can  manipulate  data  to  find  an  optimal  solution  using  multi-criteria 
evaluation  factors.  Numerous  specialized  mathematical  programming  packages,  such  as 
LINDO  (Linear  Interactive  Discrete  Optimizer),  MPSX  (IBM’s  Mathematical 
Programming  Systems  Extended),  AMPL  (A  Mathematical  Programming  Language), 
CPLEX  (first  commercial  implementation  of  simplex  method  written  in  “C”  programming 
language)  and  MathPro,  can  solve  optimization  problems  without  using  a  spreadsheet 
(ILOG  CPLEX,  LINDO,  and  IBM,  1998).  These  programs  are  typically  used  by 
researchers  and  businesses  to  solve  extremely  large  problems  that  do  not  conveniently  fit 
into  a  spreadsheet  (Ragsdale,  1997:  44).  Microsoft  Excel’s  SOLVER,  a  spreadsheet 
modeling  and  decision  analysis  add-in  option  tool  built  by  Frontline  Systems,  was  selected 
for  baseline  development  in  this  research  for  several  reasons.  The  spreadsheet  is  a 
convenient  information  technology  platform  for  baseline  design  since  the  majority  of 
price/cost  analysts  within  the  contracting  field  currently  use  spreadsheets  to  analyze 
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contractor  proposals  for  reliability  and  realism.  The  probability  of  a  contract’s  source 
selection  information  exceeding  the  capability  of  a  spreadsheet  seems  unlikely.  The 
SOLVER  program  was  also  chosen  since  it  comes  as  a  standard  option  to  the  Microsoft 
Excel  programs,  Windows  95  and  Windows  97.  The  Microsoft  Excel  package  contains 
numerous  mathematical  modeling  capabilities,  including  solving  linear,  integer  linear,  and 
nonlinear  programming  problems;  goal  programming  and  multiple  objective  optimization 
problems;  and  multi-criteria  decision-making  and  analytic  hierarchy  process  problems. 
These  features  allow  for  flexibility  and  adaptability  in  developing  the  baseline  models.  In 
addition,  Microsoft  Excel  is  a  commercially  available  software  program  which  is  currently 
used,  or  easily  obtainable,  by  most  DoD/Air  Force  contracting  offices. 

1.4  Investigative  Questions 

Two  main  investigative  questions  are  researched  in  this  thesis. 

1)  Can  a  schematic  model  be  designed  to  identify  areas  where  information 
technology  can  improve  the  flow  of  information  into  the  source  selection  evaluation 
process  and  expand  the  present  tactical  approach  to  evaluation  of  alternatives  into  a  more 
strategic  approach? 

2)  Can  a  baseline  model  be  designed,  based  upon  Microsoft  Excel 
SOLVER,  to  assist  the  SSA  in  choosing  the  optimal  or  best  alternative  source  in  a 
competitive  source  selection? 
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2.5  Overview  of  the  Thesis  Structure 

Chapter  2  surveys  literature  relevant  to  the  evaluation  of  alternative  sources  during 
the  government’s  competitive  source  selection  process.  This  process  includes  sealed 
bidding,  as  defined  in  FAR  Part  14;  competitive  proposals,  as  defined  in  FAR  Part  15;  and 
formal  source  selections,  as  defined  in  AFFARS  Appendix  AA  and  BB.  This  review  is 
essential  to  determine  the  contractual  and  legal  evaluation  boundaries  for  development  of 
the  Microsoft  Excel  SOLVER  models.  In  addition.  Chapter  2  surveys  literature  relevant 
to  finding  “strategically  focused”  information  technology  components  to  automate  the 
flow  of  information  into  the  source  selection  process  and  enhance  the  SSA’s,  and 
contracting  officials,  ability  to  make  a  better  decision  by  using  a  more  integrated, 
systematic  approach.  Chapter  2  concludes  with  a  discussion  on  the  evaluation  aspects  of 
the  Microsoft  Excel  SOLVER  program.  Chapter  3  discusses  the  general  approach  used  to 
develop  the  schematic  model  for  demonstration  of  how  information  flow  into  the  source 
selection  evaluation  process  can  be  automatically  generated  and  input  into  Microsoft  Excel 
SOLVER  for  manipulation  and  analysis  to  enhance  the  overall  source  selection  process. 
The  primary  method  of  research  for  the  development  of  the  schematic  model  was  a  case 
study  approach  to  survey  and  evaluate  primary  and  secondary  literary  sources  on 
information  technology  systems  which  could  have  the  potential  to  assist  in  the  source 
selection  decision-making  process.  The  primary  method  of  research  for  the  design  and 
development  of  the  Microsoft  Excel  baseline  models  was  a  grounded  theory  approach. 
Grounded  theory  was  chosen  as  it  allows  for  the  development  and  analysis  of  the  model 
simultaneously,  and  provides  the  researcher  the  opportunity  to  correct  deficiencies  as  they 
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occur.  Chapter  3  concludes  with  a  discussion  on  the  procedures  for  analysis  and 
verification  of  the  baseline  models.  The  baseline  models,  one  for  sealed  bidding,  one  for 
competitive  proposals,  and  one  for  AFFARS  Appendix  AA/BB  source  selections,  are 
evaluated  for  ease  of  communication,  reliability,  auditability,  and  modifiability.  Chapter  4, 
Model  Development  and  Analysis,  addresses  the  findings  of  the  data  collected  and 
analyzed  during  model  development  for  both  the  schematic  and  SOLVER  models.  An 
analysis  of  a  single  theoretical  schematic  model  is  discussed  as  it  relates  to  the  literary 
research.  A  discussion  of  the  design  and  development  of  the  Microsoft  Excel  SOLVER 
baseline  models,  observations  and  problems  which  occurred  during  development  due  to 
contracting  regulatory  constraints,  and  each  models  adaptability  to  multiple  types  of 
evaluation  criteria  based  on  communication,  reliability,  auditability,  and  modifiability  are 
addressed.  Chapter  5  contains  the  conclusions  and  recommendations  for  further  research 
resulting  from  the  study  of  the  problems  identified  in  this  thesis. 
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2.  Literature  Review 


This  chapter  is  divided  into  three  main  sections,  1)  A  survey  of  literature 
concerning  the  competitive  source  selection  process,  specifically  the  basis  of  award  in 
various  types  of  government  acquisitions;  2)  A  survey  of  literature  concerning 
information  technology  components  and  tools  useful  for  collecting  and  flowing 
information  into  the  source  selection  process  for  manipulation  and  analysis  during  final 
evaluation  to  allow  for  more  informed  decisions;  and  3)  A  survey  of  literature  discussing 
Microsoft  Excel  SOLVER  as  the  model  base  for  the  manipulation  and  analysis  of  the 
source  selection  information  flowing  from  the  schematic  model  into  the  decision-making 
process.  Microsoft  Excel  features,  such  as  scoring  models  for  multi-criteria  decision¬ 
making,  the  analytical  hierarchy  process,  mathematical  modeling  and  optimization  tools, 
and  decision  analysis  techniques  are  discussed  in  relation  to  the  source  selection  process 
and  the  constraints  placed  upon  the  model  due  to  contracting  policy  and  regulations.  This 
information  is  important  to  understand  the  mechanics  behind  the  Microsoft  Excel 
SOLVER  software  program.  Chapter  2  concludes  with  a  discussion  of  how  the  source 
selection  policies  and  regulations,  the  flow  of  information  into  the  source  selection 
process,  and  the  manipulation  and  analysis  of  the  data  using  the  Microsoft  Excel  SOLVER 
program  can  benefit  the  Government  by  shifting  from  a  tactical  focus  to  a  more  strategic 
focus  on  decision-making. 
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2.1  Acquisition  Overview  and  Brief  Explanation  of  Key  Terms 

2. 1.1  Competitive  Source  Selection  Process.  The  two  primary  competitive 
procedures  are  sealed  bidding  and  competitive  proposals.  Competitive  procedures  in  the 
source  selection  process  entail  the  use  of  “full  and  open  competition.”  The  Office  of 
Federal  Procurement  policy  Act,  41  U.S.C.  403  (7),  defines  “full  and  open  competition  to 
mean  all  responsible  sources  are  permitted  to  submit  sealed  bids  or  competitive  proposals 
on  a  procurement”  (Cibinic  and  Nash,  1986:  288).  In  addition,  full  and  open  competition 
may  be  limited  by  excluding  particular  sources,  such  as  in  small  business  or  labor-surplus 
set-asides;  or  when  the  agency  head  determines  to  exclude  an  incumbent  contractor  to 
establish  or  maintain  alternative  sources  (FAR  6.3).  Seven  specific  types  of  procurements 
are  authorized  by  statute  to  use  other  than  full  and  open  competition.  Contracts  using 
procedures  under  these  seven  exceptions,  as  defined  in  FAR  6.302,  are  not  incorporated 
into  the  context  of  this  research.  These  seven  types  of  procurements  are  considered  non¬ 
competitive,  and  usually  have  only  a  single  offeror  (or  alternative)  proposing  for  award  of 
the  contract.  These  procurements  use  negotiated  procedures  and  are  considered  sole 
source  acquisitions.  For  the  purposes  of  this  investigation,  only  procurements  utilizing  full 
and  open  competition,  or  limited  competition,  are  analyzed.  Limited  competition  includes 
acquisitions  in  which  competition  is  restricted  due  to  a  small-business  or  a  small 
disadvantaged  business  set-aside. 

2.2  Basis  for  Contract  Award 

Contract  award  in  a  Government  acquisition  is  based  upon  the  complexity  of  the 
acquisition  and  the  need  to  discuss  a  offeror’s  proposal.  In  non-complex  acquisitions, 
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which  do  not  require  discussions,  award  can  be  made  to  the  lowest  priced  offeror  which 
has  met  the  materia!  requirements  of  the  invitation  for  bid;  the  offeror  has  proposed 
exactly  what  was  asked  for  without  any  significant  deviations.  When  the  complexity  of 
the  acquisition  increases  to  warrant  discussions,  the  SSA  has  more  options  to  select  from 
as  the  basis  of  award.  Award  can  be  made  on  the  basis  of  selecting  the  lowest  priced, 
technically  acceptable  offeror,  in  which  the  proposals  are  first  evaluated  to  determine 
which  offerors  have  met  the  minimum  technical  requirements  of  the  solicitation,  and  award 
goes  to  the  lowest  priced  offeror  whose  proposal  meets  the  minimum.  A  second  option 
available  to  the  SSA  is  to  award  to  the  offeror  whose  proposal  is  determined  to  be  the  best 
value  to  the  Government.  As  with  the  lowest  cost,  technically  acceptable  basis  for  award, 
the  basis  for  a  best  value  award  first  determines  which  offerors  are  technically  acceptable 
based  upon  the  minimum  requirements  stated  in  the  solicitation.  However,  using  the  best 
value  basis  for  award,  the  SSA  has  the  latitude  to  chose  a  higher  priced  proposal  if  its 
increase  in  cost  can  be  justified  by  its  technical  superiority  in  relation  to  the  lower  priced 
proposals  within  the  technically  acceptable  competitive  range.  Three  basic  source 
selection  procedures  are  discussed  in  the  next  sections  in  relation  to  their  basis  for  award. 
These  include  sealed  bidding,  as  defined  in  FAR  Part  14;  Competitive  Proposals,  as 
defined  in  FAR  Part  15;  and  Formal  Source  Selections,  as  defined  in  AFFARS  Appendix 
AA/BB. 

2.2.1  Sealed  Bidding  (FAR  Part  14).  With  sealed  bidding  techniques,  the  basis 
for  award  is  to  evaluate  the  offers  and  award  to  the  responsible  offeror  whose  bid  is  the 
most  advantageous  to  the  Government,  considering  only  price  and  the  price-related 


factors  (FAR  14. 101(d)(e)).  In  essence,  the  award  goes  to  the  overall,  lowest  priced 
offeror  whose  bid  conforms  to  the  requirements  of  the  invitation  for  bids  (IFB). 

Depending  upon  the  circumstances  of  the  procurement,  the  evaluation  process  to 
determine  which  offeror  is  offering  the  lowest  cost  to  the  Government  varies  from  a 
simple  to  a  complex  task.  The  complexity  of  the  evaluation  increases  in  relation  to  the 
number  of  items  being  procured,  and  whether  a  single  aggregate  award  or  multiple  awards 
will  be  issued.  In  addition,  price  adjustment  clauses  allowing  for  discounts  or  economic 
price  adjustments  increase  the  complexity  of  the  award  evaluation.  When  the  evaluation 
criteria  contain  “price-related  factors,”  such  as  transportation  costs,  life  cycle  costs,  taxes, 
options,  or  Buy  American  Act  provisions,  a  thorough  examination  of  the  bids  to  determine 
the  most  advantageous  award  can  become  quite  complicated.  The  contracting  official 
must  also  analyze  the  bids  for  unbalanced  bidding  (bidding  high  on  some  items  and  low  on 
others)  or  bids  which  are  front  end  loaded  (bids  which  enable  an  offeror  to  recover  money 
in  advance  of  the  work  performed  by  bidding  high  on  the  earliest  delivered  work  and  low 
on  work  delivered  last)  (Cibinic  and  Nash,  1986:  442-443).  As  previously  stated,  the 
award  can  be  made  on  an  “all-or-none”  or  aggregate  basis  or  can  be  divided  into  multiple 
awards  if  it  is  determined  to  be  economically  advantageous  to  the  Government  and  the 
provision  at  FAR  52.214-22,  “Evaluation  of  Bids  for  Multiple  Awards”  is  included  in  the 
IFB.  Some  offerors  propose  various  combinations  of  split  awards  which  must  be  taken 
into  consideration.  For  evaluation  purposes,  when  making  a  determination  to  issue  a 
single  or  make  multiple  awards,  a  $500  administrative  fee  is  added  to  the  costs  of  each 
individual  contract  to  be  awarded  under  the  IFB  if  multiple  awards  appear  feasible.  This 
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$500  fee  is  a  predetermined  cost  the  Government  will  incur  due  to  the  additional  costs  of 
issuing  and  administering  multiple  contracts.  Individual  awards  are  based  upon  the  items 
or  combination  of  items  that  result  in  the  lowest  aggregate  cost  to  the  Government, 
including  the  assumed  administrative  fees  (FAR  14.201-8  (c)).  The  contracting  official 
must  meticulously  evaluate  all  proposals  to  determine  the  most  advantageous  award 
alternative  for  the  Government,  and  must  complete  this  task  within  the  bid  evaluation 
time. 


2.2.2  Competitive  Proposals  (FAR  Part  15,  and  AFFARS  Appendix  AA/BB). 

2.2.2. 1  General  Information.  In  competitive  acquisitions  using 
negotiated  procurement  procedures,  the  intent  is  to  minimize  the  complexity  of  the 
solicitation,  evaluation,  and  the  source  selection  decision-making  process,  using 
procedures  designed  to  encourage  impartial  and  comprehensive  evaluations  of  proposals. 
The  intent  is  to  use  processes  and  procedures  allowing  for  the  selection  of  the  proposal 
representing  the  best  value  to  the  Government  (FAR  15.002  (b)).  .fAR  2. 101  defines 
“best  value”  as  “the  expected  outcome  of  an  acquisition  that,  in  the  Government's 
estimation,  provides  the  greatest  overall  benefit  in  response  to  the  requirement”  (FAR 
2.101).  According  to  FAR  15.304,  the  SSA  has  broad  discretion  in  selecting  the 
evaluation  factors  and  subfactors  that  apply  to  an  acquisition  but  must  include  price  or 
cost,  quality,  and  at  least  consider  past  performance  as  evaluation  factors.  A  complete 
citation  regarding  evaluation  factors  and  significant  subfactors  for  competitive  negotiated 
acquisitions  (FAR  15.304)  can  be  seen  in  Appendix  A. 
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2. 2. 2. 2  FAR  Part  15,  Competitive  Proposals.  The  use  of  competitive 
proposals  lends  flexibility  to  the  source  selection  evaluation  process.  Award  can  be  made 
on  the  basis  of  lowest-priced  offeror,  lowest-priced,  technically  acceptable  offeror,  or  best 
value  to  the  Government.  If  the  acquisition  is  noncomplex  so  as  to  warrant  award  without 
discussions  to  the  lowest-priced  offeror,  generally  sealed  bidding  procedures  are  used. 
Whichever  method  is  used,  it  must  be  published  in  the  solicitation,  along  with  the 
evaluation  criteria,  so  all  potential  offerors  will  know  how  their  proposals  are  being  judged 
(FAR  15). 


2. 2. 2. 2.1  Decisional  Rules.  Various  systems  are  used  to 
determine  the  measures  of  relative  importance  for  the  criteria  selected  for  evaluation 
(Cibinic  and  Nash,  1986:  548).  Two  decision  rule  techniques  are  used  to  accomplish  this 
task.  These  are  general  guidelines  on  the  relative  importance  of  various  criteria  or 
developing  a  mathematical  system.  DARCOM  Pamphlet  715-3,  dated  24  Oct  80, 
paragraph  4-4,  states: 

Relativity  of  each  criterion  to  the  total  mission  requirement 
must  be  established  in  order  to  summarize  the  scores.  The 
decisionmaker  may  do  this  by  weighting  the  criteria,  by  priority  or 
trade-off  statements,  by  judgmental  decision  rules,  or  a  combination 
of  these.  Weighting  would  involve  an  assignment  of  relative 
importance  among  the  criteria  by  breaking  up  a  constant  sum 
(typically  100  or  1000  points).  For  source  officials  who  do  not 
want  to  specify  numerical  relationships,  the  criteria  may  be  related 
by  priority  statements  and  the  scores  combined  by  judgment  using 
these  priorities  rather  than  numeric  formulas  using  weights.  For 
example,  in  a  priority  statement  “cost”  may  be  said  to  be  slightly 
more  important  than  “management”  but  slightly  less  than 
“operational  suitability.”  A  decision  rule  would  tell  how  to  deal 
with  a  factor  under  varying  conditions.  For  example,  a  decision 
rule  might  be  “if  management  is  rated  anything  less  than 
satisfactory,  the  entire  proposal  is  unacceptable,”  or  “if  the 
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proposed  price  is  30-percent  higher  than  the  Government  estimate, 
it  will  be  judged  as  being  potentially  unrealistic,  the  cost  score  will 
be  penalized,  and  the  technical  proposal  will  be  reevaluated  to  see  if 
there  is  some  misunderstanding  of  the  requirements.”  In  order, 
then,  to  provide  a  base  for  the  evaluation,  a  precise  definition  of 
each  established  criterion  must  be  prepared  in  narrative  form  to 
indicate  what  it  is  and  how  it  is  to  be  used,  and  a  description  of  the 
alpha  or  numeric  standard  indicating  the  desired  performance  for 
each  major  element,  and  the  relationship  among  factors  and 
subfactors.  Definitions  should  be  either  included  in,  or  appended  to 
the  selection  plan.  (Cibinic  and  Nash,  1986:  548) 

The  SSA  can  choose  which  decisional  process  to  use  for  evaluation.  The  goal  is  to 
choose  a  process  providing  a  sound  and  logical  decision.  A  system  using  a  generalized 
priority  statement  provides  flexibility  in  the  evaluation,  while  a  purely  mathematical 
weighting  system  is  quite  restrictive.  Systems  are  composed  of  one  or  more  of  the 
following  decisional  rules:  Fixed  Weights;  Variable  Weights;  Tradeoff  Analysis;  Go,  No- 
Go;  and  Lowest-Priced  Acceptable  Proposal  (Cibinic  and  Nash,  1986:  548-554). 

2. 2. 2. 2. 1.1  Fixed  Weights.  In  this  method,  fixed  weights 
are  assigned  or  attributed  to  each  evaluation  factor.  The  simplest,  and  most  common 
method  of  assigning  fixed  weights  is  by  percentage  distribution.  Percentage  distribution 
distributes  weights  among  the  major  areas  and  elements  based  upon  a  100  percent  total 
score.  Another  common  mathematical  system  is  the  point  system.  The  point  system 
assigns  points  on  the  basis  of  importance  of  the  criteria.  Table  1  compares  the  percentage 
distribution  system  and  the  point  system. 

Variations  of  the  fixed  weight  system  are  also  used.  One  example  of  a  variation  is 
to  determine  the  relative  weight  or  importance  of  the  evaluation  factors  in  relation  to  their 
expected  impact  on  life  cycle  costs  (Cibinic  and  Nash,  1986:  550-551). 
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Table  1 


Fixed  Weights  System 

An  Example  of  Percentage  Distribution  and  Point  System 
_ (Cibinic  and  Nash,  1986;  549-550) _ 


Evaluation 

Percent^ 

Point 

Criteria 

Distribution 

Svstem 

I.  Adequacy  of  Technical  Proposal 

60% 

200  pts. 

a  literature  Search  and 

investigation  methodology 

20% 

100  pts. 

b.  Proposed  sources  of  information 

15% 

35  pts. 

c.  Han  for  Assessing  Literature 

15% 

35  pts. 

d  Presentation  ofTindirgs 

10% 

30  pts. 

IL  Persomd  Qualifications 

40% 

100  pts. 

a  Technical  Experience  of 

25% 

70  pts. 

project  staff 

b.  Educational  Qualifications 

10% 

20pts.  1 

c.  Qialifications  of  Consultants 

5% 

10  pts. 

2. 2. 2. 2. 1.2  Variable  Weights.  In  certain  situations,  the 
SSA  requires  more  flexibility  in  the  weighting  of  factors  than  fixed  weight  affords. 
Variable  weights  are  commonly  used  in  situations  where  the  SSA  decides  that  cost  or 
price  should  remain  flexible  to  become  more  important  when  technical  proposals  are  of 
relatively  equal  weight  (Cibinic  and  Nash,  1986:  551). 

2. 2. 2. 2. 1. 3  Tradeoff  Analysis.  T  radeoff  analysis  evaluates 
differences  in  technical  and  management  proposals  to  determine  whether  or  not  the 
differences  justify  paying  a  higher  cost  or  price.  Ordinarily,  technical  and  management 
factors  are  scored  using  a  point  system,  and  the  differences  between  the  technical 
proposals  are  compared  to  the  differences  in  the  cost/price  proposals  to  measure  the 
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cost/benefit  of  a  tradeoff.  Tradeoff  analysis  allows  an  award  on  the  basis  of  best  value 
rather  than  lowest  cost.  If  a  solicitation  is  silent  on  the  relative  importance  of  cost  in 
relation  to  the  technical  criteria,  then  the  cost  and  technical  criteria  are  determined  to  have 
equal  importance.  The  SSA  has  great  latitude  in  determining  the  use  of  technical  and  cost 
evaluation  information.  The  SSA  can  make  cost/technical  tradeoffs  which  are  consistent 
and  rational  based  upon  the  established  evaluation  factors  in  the  solicitation.  Cost  factors 
are  not  scored  because  the  relative  weight  in  a  cost/technical  tradeoff  can  be  determined 
only  after  the  contracting  official  has  analyzed  the  relative  worth  and  significance  of  the 
technical  proposals.  FAR  15.101-1  states  the  following  regarding  the  tradeoff  process: 

(a)  A  tradeoff  process  is  appropriate  when  it  may  be  in  the  best 
interest  of  the  Government  to  consider  award  to  other  than  the  lowest 
priced  offeror  or  other  than  the  highest  technically  rated  offeror. 

(b)  When  using  a  tradeoff  process,  the  following  apply: 

(1)  All  evaluation  factors  and  significant  subfactors  that  will  affect 
contract  award  and  their  relative  importance  shall  be  clearly  stated  in 
the  solicitation;  and 

(2)  The  solicitation  shall  state  whether  all  evaluation  factors  other 
than  cost  or  price,  when  combined,  are  significantly  more  important 
than,  approximately  equal  to,  or  significantly  less  important  than  cost 
or  price. 

(c)  This  process  permits  tradeoffs  among  cost  or  price  and  non-cost 
factors  and  allows  the  Government  to  accept  other  than  the  lowest  priced 
proposal.  The  perceived  benefits  of  the  higher  priced  proposal  shall 
merit  the  additional  cost,  and  the  rationale  for  tradeoffs  must  be 
documented  in  the  file.  (FAR  15.101-1) 

2. 2. 2. 2. 1.4  Go,  No-Go.  In  a  Go,  No-Go  situation, 
the  criteria  are  evaluated  on  a  “pass/fail”  or  “adequate/inadequate”  basis.  A 
passing  or  adequate  score  means  the  offeror  has  met  the  minimum  requirement  for 
selection.  An  example  which  uses  Go,  No-Go  techniques  is  award  to  the  lowest 
cost,  technically  acceptable  offeror.  The  technical  proposal  is  first  evaluated  on  a 
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pass/fail  basis,  and  award  is  then  made  to  the  offeror  with  the  lowest  cost/price 
who  has  a  passing  score  (Cibinic  and  Nash,  1986:  553-554). 

2. 2. 2. 2. 1. 5  Lowest-Priced  Acceptable  Proposal 
This  technique  is  similar  to  the  Go,  No-Go  method  in  that  it  utilizes  the  Go,  No- 
Go  techniques  by  applying  them  to  the  entire  technical  and  management  proposal. 
After  evaluation,  unacceptable  proposals  may  still  be  included  within  the 
competitive  range,  and  given  a  second  chance  to  become  acceptable.  Award  is 
made  to  the  acceptable  offeror  with  the  lowest  evaluated  cost/price  (Cibinic  and 
Nash,  1986:  554).  FAR  15.101-2  states  the  following  regarding  the  lowest  price 
technically  acceptable  source  selection  process: 

(a)  The  lowest  price  technically  acceptable  source  selection  process  is 
appropriate  when  best  value  is  expected  to  result  from  selection  of  the 
technically  acceptable  proposal  with  the  lowest  evaluated  price. 

(b)  When  using  the  lowest  price  technically  acceptable  process,  the 
following  apply: 

(1)  The  evaluation  factors  and  significant  subfactors  that  establish  the 
requirements  of  acceptability  shall  be  set  forth  in  the  solicitation. 
Solicitations  shall  specify  that  award  will  be  made  on  the  basis  of  the 
'lowest  evaluated  price  of  proposals  meeting  or  exceeding  the 
acceptability  standards  for  non-cost  factors.  If  the  contracting  officer 
documents  the  file  pursuant  to  15.304(c)(3)(iii),  past  performance  need 
not  be  an  evaluation  factor  in  lowest  price  technically  acceptable 
source  selections.  If  the  contracting  officer  elects  to  consider  past 
performance  as  an  evaluation  factor,  it  shall  be  evaluated  in  accordance 
with  15.305.  However,  the  comparative  assessment  in  15.305(a)(2)(i) 
does  not  apply. . . . 

(2)  Tradeoffs  are  not  permitted. 

(3)  Proposals  are  evaluated  for  acceptability  but  not  ranked  using  the 
non-cost/price  factors. 

(4)  Exchanges  may  occur. 

2. 2. 2. 2. 2  Scoring  Methods.  Scoring  methods  prescribe  the 
procedures  used  to  compare  each  offeror’s  proposal  against  the  Government’s 
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requirements  and  to  rate  them  in  relation  to  competitor  s  proposals.  The  method  of 
comparison  need  only  have  a  rational  basis  and  be  applied  in  a  good  faith  manner  (Cibinic 
and  Nash,  1986:  555).  According  to  FAR  15.305(a),  evaluations  may  be  conducted  using 
any  rating  method  or  combination  of  methods,  including  color  or  adjectival  ratings, 
numerical  weights,  and  ordinal  rankings.  The  guidance  on  scoring  methods  varies  among 
Government  agencies.  DoD  Directive  4105.62  requires  a  narrative  statement  as  part  of 
the  evaluation,  but  the  regulation  is  adaptable  as  to  the  scoring  techniques  used.  DoD 
Directive  4105.62,  paragraph  E.4.c.  states: 

There  is  no  prescribed  methodology  for  rating.  Past 
practices  include  color  coding,  numerical,  and  plus  or  minus  checks. 

The  important  thing  is  not  the  rating  methodology  but  the 
consistency  with  which  it  is  applied  to  elements  of  proposals  and 
among  proposals  to  ensure  a  thorough  and  fair  evaluation. 

Evaluators  must  be  weli  rounded  in  their  field  of  expertise  and  be 

able  to  apply  mature  professional  judgment . evaluators  must 

support  the  rating  assigned  with  a  concise  narrative  that  addresses 
strengths,  weaknesses,  and  risks  in  the  proposal.  (Cibinic  and 
Nash,  1986:  555-556) 

Various  types  of  scoring  systems  include  adjective  ratings;  numerical 
scores;  narratives;  and  ranking  (Cibinic  and  Nash,  1986:  555-556). 

22.22.21  Adjectival  Ratings.  Adjective  ratings 
describe  the  quality  of  a  proposal  in  terms  of  words  or  symbols  representing 
words.  All  adjectival  rating  systems  compare  the  proposal  against  the  contract 
requirements  and  provide  information  using  the  words  or  symbols  to  describe  the 
extent  of  compliance  or  problems  represented.  The  Air  Force  uses  a  color  coding 
system,  a  descriptive  adjective  system,  and  a  symbol  system  (described  in 
paragraph  2.2.2. 3.1  below)  as  part  of  its  formal  source  selection  procedures.  A 
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number  of  source  selection  manuals  suggest  using  adjectival  systems  for  presenting 
source  selection  evaluation  details  to  the  SSA,  even  if  another  type  of  scoring 
system  was  used  for  evaluation  (Cibinic  and  Nash,  1986:  558). 

2. 2. 2. 2. 2. 2  Numerical  Scores.  Numerical  point 
scores  may  be  used  to  evaluate  proposals.  The  Air  Force  discourages  the  use  of 
numerical  weighting  of  evaluation  criteria  in  its  formal  source  selection  procedures 
due  to  difficulty  supporting  modest  differences  in  technical  merit  (AFFARS  BB- 
304  (c));  for  example,  it  is  difficult  to  measure  and  support  an  offeror’s  proposal  as 
being  49%  or  5 1%  better  than  an  evaluation  criteria  when  the  standard  for 
acceptability  is  50%.  The  validity  of  evaluation  by  numerical  point  scores  may  be 
scrutinized.  In  a  decision  by  the  Comptroller  General,  in  Bunker  Ramo 
Corporation,  56  Vomp.  Gen.  712  (1977),  77-1  CPD  427,  the  Comptroller  stated: 

Numerical  point  scores,  when  used  for  proposal  evaluation, 
are  useful  as  guides  to  intelligent  decision  making,  but  are  not 
themselves  controlling  in  determining  award,  since  these  scores  can 
only  reflect  the  disparate,  subjective,  and  objective  judgments  of  the 
evaluators.  Whether  a  given  point  spread  between  competing 
offers  indicates  the  significant  superiority  of  one  proposal  over 
another  depends  on  the  facts  and  circumstances  of  each 
procurement,  and  while  technical  scores  must  of  course  be 
considered  by  SSAs,  such  officials  are  not  bound  thereby.  (Cibinic 
and  Nash,  1986:  559) 

There  are  numerous  variations  of  numerical  scoring  systems.  Normally 
numerical  scoring  systems  assign  a  score  value  ranging  from  0  to  10.  Table  2 
describes  a  cross  reference  between  a  sample  numerical  and  adjectival  scoring 
system. 
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Table  2 


Sample  Cross  Reference  of 
Numerical  and  Adjectival  Scoring  Systems 
(Cibinic  and  Nash,  1986:  559-560) 


Numerical 

Adjective 

Definition/ 

Score 

Rating 

Evaluation 

10 

Excellent— 

(innovative,  comprehensive  and  complete 
in  all  details,  meets  all  requirements  and 
objectives  without  “gold  plating”) 

9 

Very  Good— 

(substantial  response  in  clearly  definable 
detail,  meets  all  critical  requirements) 

7 

Average— 

(generally  meets  minimum  requirements) 

6 

Poor- 

(lack  of  essential  information  to 
substantiate  data  presented) 

5 

Unsatis¬ 

factory— 

(lack  of  understanding  of  requirements  or 
omissions  in  major  areas) 

0 

No  data— 

In  addition.  Comptroller  General  decisions  have  discouraged  the  use  of  procedures 
which  “distort”  scores.  A  system  which  assigns  scores  by  using  a  formula  runs  the  risk  of 
distorting  evaluation  results.  Two  examples  of  numerical  scoring  methods  that  tend  to 
distort  data  are  “spread  scoring,”  which  involves  assigning  maximum  points  to  the  offeror 
with  the  lowest  cost  and  zero  points  to  the  offeror  with  the  highest  costs;  and 
mathematical  or  statistical  analysis  techniques,  which  assign  maximum  points  to  the 
offerors  whose  proposed  costs  are  closest  to  the  Government  estimate  and  to  penalize 
offerors  in  relation  to  the  amount  of  deviation  their  costs  are  from  the  estimate.  Since  this 
penalty  was  applied  both  above  and  below  the  Government  estimate,  the  second  method 
tends  to  penalize  the  lowest  cost/priced  offeror  (Cibinic  and  Nash,  1986:  561). 
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2.2.2.2.23  Narrative.  Narrative  statements  are  required  by 
DoD  Directive  4105.62.  A  narrative  evaluation  must  be  used  in  conjunction  with  other 
scoring  methods  to  indicate  strengths  and  weaknesses.  The  goal  of  the  narrative 
evaluation  is  to  enhance  communication  on  specific  information  regarding  relative 
advantages  and  disadvantages  to  the  SSA  (Cibinic  and  Nash,  1986:  561). 

2.2. 2. 2.2.4  Ranking.  The  direct  ranking  of  proposals, 
without  the  aid  of  numerical  scores,  has  been  upheld  by  Comptroller  General  Decision  B- 
205380,  82-2  CPD  37  (1982)  for  Development  Associates  Inc.  Numerical  scores  do  not 
“transform  the  technical  evaluation  into  a  more  objective  process”  (Cibinic  and  Nash, 

1986:  561). 

2.2.2. 3  AFFARS  Appendix  AA/BB  Source  Selections.  The  same  rules 
and  regulations  as  described  in  paragraph  2.2. 2. 2  above  govern  acquisitions  using 
AFFARS  Appendix  AA  and  BB,  however  these  appendices  supplement  the  information 
provided  in  the  FAR  to  require  a  more  stringent  process  on  the  AF.  AF  acquisitions  over 
$5  million  are  required  to  use  a  more  formal  process  of  source  selection.  Depending  on 
the  dollar  value  and/or  the  criticality  of  the  acquisition,  either  Appendix  AA  or  Appendix 
BB  will  be  used.  Appendix  AA  has  more  steps  in  the  overall  process  and  higher  visibility, 
but  the  evaluation  processes  for  Appendix  AA  and  BB  are  essentially  the  same.  Both 
procedures  establish  their  evaluation  criteria  on  a  hierarchical  basis  and  use  an  adjective 
rating  system  which  implements  a  color  coding  system. 
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2. 2. 2. 3. 1  Adjectival  Coding  System. 


2. 2. 2. 3. 1.1  Colors.  The  color  coding  system  for  Essential 
Characteristics  or  Baseline  Requirements  is  described  in  Table  3. 

Table  3 


AF  Color  Rating  System 

Color  Coding  for  Essential  Characteristics  or  Baseline  Requirements 
(AFFARS  Appendix  BB-304)  and  *(Cibinic  and  Nash,  1986:  558) 


COLOR 

at  r - 

RATING 

DEFINITIONS 

Blue 

Exceptional 

Exceeds  specified  performance  or  capability 
in  a  beneficial  way  to  the  Air  Force;  high 
probability  of  success*;  no  significant 
weaknesses. 

Green 

Acceptable 

Meets  standards;  good  probability  of 
success*;  weaknesses  can  be  readily 
corrected. 

Yellow 

Marginal 

Fails  to  meet  standards;  low  probability  of 
success*;  however,  significant  deficiencies 
are  correctable. 

Red 

Unacceptable 

Fails  to  meet  a  minimum  requirement  of  the 

RFP  and  the  deficiency  is  uncorrectable 
without  a  major  revision  of  the  proposal. 

Recent  policy  changes  have  led  to  the  segregation  of  “Essential  Characteristics  or 


Baseline  Requirements”  from  “Minimum  Mandatory  Requirements”  for  determining  the 
competitive  range.  As  a  result,  there  are  now  two  separate  color  coding  schemes  for 
evaluation.  Minimum  Mandatory  Requirements  are  the  initial  pass/fail  criteria  used  to 
reduce  the  number  of  proposals  evaluated  in  depth  by  eliminating  offerors  from  the 
competitive  range.  The  color  coding  scheme  for  the  Essential  Characteristics  is  shown  in 
Table  3;  the  color  coding  scheme  for  the  minimum  mandatory  requirements  is  a  pass/faii 


or  a  go/no  go  evaluation,  since  the  offeror  is  evaluated  as  either  green  for  acceptable,  met 

the  minimum  requirement  or  red,  does  not  meet  the  minimum  requirement.  A  source 

selection  may  have  essential  characteristics  or  baseline  requirements  or  a  combination  of 

minimum  mandatory  requirements  and  essential  characteristics  or  baseline  requirements 

(AFFARS  Appendix  BB-203  (2)(i)).  AFFARS  Appendix  BB-304(i)>  states: 

If  a  source  selection  has  a  mix  of  minimum  mandatory  requirements 
and  essential  characteristics  or  baseline  requirements,  most  likely  an 
initial  competitive  range  will  be  determined  on  the  basis  of  cost 
(price)  and  other  factors  that  were  stated  in  the  solicitation  and  shall 
include  all  proposals  that  have  a  reasonable  chance  of  being  selected 
for  award.  The  minimum  mandatory  portion  need  not  be  color  rated; 
however,  if  it  is,  it  shall  be  displayed  as  "green"  for  meeting  the 
requirement  or  "red"  for  not  meeting  the  minimum  mandatory  requirement. 

A  narrative  identifying  the  deficiencies  is  sufficient.  The  essential 
characteristics  or  baseline  requirements  are  usually  then  evaluated  ana 
color  rated.  This  minimizes  the  work  of  the  technical  team  by  not 
requiring  them  to  evaluate  the  essential  characteristics  or  baseline 
requirements  for  offerors  which  did  not  meet  the  minimum  mandatory 
requirements. 

However,  not  all  source  selections  will  utilize  minimum  mandatory  requirements. 

2.2.2. 3. 1.2  Symbols.  Symbols  may  be  used  for  ratings  at 
the  element  level.  For  example,  a  plus  (+)  sign  may  indicate  that  the  offeror  has  exceeded 
a  standard;  a  check  (V)  may  indicate  that  the  offeror  has  met  the  standard;  and  a  minus  (-) 
sign  may  indicate  that  the  standard  has  not  been  met  for  the  evaluated  element  (AFFARS 
Appendix  BB-304(e)). 


2.2.2.3.1.3  Descriptive  Adjectives.  The  AF  assesses  risk 
by  assigning  a  risk  rating,  normally  at  the  item  summaiy  level.  Two  types  of  risk  assessed 
by  the  AF  include  Proposal  Risk  and  Performance  Risk  (see  definitions  in  next 
subsection).  Table  4  provides  a  description  of  the  risk  ratings  and  their  definitions. 
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In  summary,  any  scoring  method  described  in  paragraph  2. 2.2. 2  could  be  utilized. 
The  Air  Force  has  implemented  the  color  adjectival  rating  system  as  its  standard  method 
of  evaluation  (AFFARS  Appendix  BB-304).  AFFARS  BB-304(c)  discourages  the  use  of 
numerical  weighting  of  evaluation  criteria  because  of  the  difficulty  to  support  small 
differences  in  technical  merit  when  distinguishing  differences  in  technical  proposals 
(AFFARS  BB-304  (c)). 


Table  4 

Definitions  of  Risk  Ratings 
(AFFARS  Appendix  BB) 

RISK 

DEFINITION 

High 

Likely  to  cause  significant  serious  disruption  of  schedule,  increase 
in  cost,  or  degradation  of  performance  even  with  special  contractor 
emphasis  and  close  government  monitoring. 

Moderate 

Can  potentially  cause  some  disruption  of  schedule,  increase  in  cost, 
or  degradation  of  performance.  However,  special  contractor 
emphasis  and  close  government  monitoring  will  probably  be  able  to 
overcome  difficulties. 

Low 

Has  little  potential  to  cause  disruption  of  schedule,  increase  in  cost, 
or  degradation  of  performance.  Normal  contractor  effort  and 
normal  government  monitoring  will  probably  be  able  to  overcome 
difficulties. 

2.2.23.2  Hierarchical  Basis  for  Evaluation  Criteria.  The  AF  has 
broad  discretion  in  determining  the  specific  number  and  types  of  evaluation  criteria  it 
selects  to  use  on  any  given  acquisition.  According  to  Comptroller  General  Decision  B- 
186614,  76-2  CPD  235  (1976)  in  a  legal  case  with  Augmentation,  Inc.,  “The 
determination  of  an  agency’s  minimum  needs  and  the  selection  and  weights  of  evaluation 
criteria  to  be  used  to  measure  how  well  offerors  will  meet  those  needs  are  within  the  broad 


28 


discretion  entrusted  to  agency  procurement  officials”  (Cibinic  and  Nash,  1986:  541). 
Three  common  areas  of  evaluation  include  Technical  Aspects,  Cost/Price,  and  Past 
Performance.  “Air  Force  source  selection  awards  are  based  on  an  integrated  assessment 
of  each  offeror’s  proposal  using  factors  and  subfactors  which  include:  cost  (price) 
criterion,  specific  criteria,  assessment  criteria,  proposal  risk,  performance  risk,  and  general 
considerations”  (AFFARS  Appendix  BB-203(a)).  Evaluation  criteria  consist  of  three 
types:  specific  criteria,  cost  (price)  criterion,  and  assessment  criteria  (AFFARS  Appendix 
BB-203(b)).  The  relative  importance  of  the  cost  (price)  criterion,  specific  criteria,  and 
general  considerations  are  stated  in  the  solicitation  (AFFARS  Appendix  BB-204(c)(5)). 

2. 2. 2. 3. 2.1  Specific  Criteria.  The  specific  criteria  relate  to 
the  particular  program  or  requirements  characteristics  and  what  the  offeror  has  proposed 
(AFFARS  Appendix  BB-203(2)).  A  list  of  key  characteristics  is  developed  that 
distinguish  good  performance  from  poor  performance.  This  list  includes  the  key 
discriminators  forming  the  basis  for  all  evaluation  criteria.  These  criteria  should  include 
those  things  considered  important  to  the  customer  about  the  specific  requirement,  such  as 
quality  of  service,  environmental  considerations,  and  management  (AFFARS  Appendix 
BB-203).  The  specific  criteria  are  typically  divided  into  technical  and/or  management 
evaluation  areas.  Examples  of  specific  criteria  include  major  areas  of  performance, 
general  facility  maintenance,  control  of  Government  assets,  and  management  (AFFARS 
Appendix  BB-203 (2)). 

Specific  criteria  are  segregated  into  tiering  levels  to  facilitate  the  rating  process. 
The  hierarchical  basis  of  the  specific  criteria  described  in  the  AF  Supplement  involves 
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establishment  of  Area,  Factor,  Subfactor  and  Element  levels,  listed  in  order  of  the  most 

broad  category  to  the  most  specific  subcategory  (AFFARS  Appendix  BB-304  and  Cibinic 

and  Nash,  1986:  541).  The  level  of  subdivision  depends  on  the  complexity  of  the  area 

being  evaluated.  Factors  are  related  to  characteristics  which  are  important  to 

successful  contract  performance.  According  to  AFFARS  Appendix  BB-304(b): 

Color  ratings  are  mandatory  at  the  factor  and  subfactor  level.  Colors  may 
also  be  used  at  the  element  level,  although  symbols  may  be  used  as  an 
alternative  at  these  lower  levels.  The  color  rating  depicts  how  well  each 
offeror  meets  the  evaluation  standards.  Color  ratings  are  not  summarized 
above  the  factor  level.  However,  if  the  SSA  requires  a  summary  rating  at 
the  area  level  in  the  SSP,  color  ratings  shall  be  used.  To  provide  for  a 
standard  color  scheme,  the  spectrum  (in  Table  1)  shall  be  used.  Ratings 
must  be  accompanied  by  a  consistent  narrative  assessment  (inclusive  of 
strengths  and  weaknesses)  of  the  basis  for  the  rating.  The  definitions 
stated  below  shall  be  followed  any  time  color  ratings  are  used.  Deviations 
from  these  definitions  can  only  be  obtained  by  forwarding  a  request 
through  SAF/AQCP. 

If  an  offeror’s  proposal  is  evaluated  as  unacceptable  or  “red”  at  any  level  of  the 
evaluation  criteria,  this  event  is  recorded  in  the  rating  and  narrative  assessment  at  that 
level  and  at  each  higher  level  in  the  evaluation  process.  This  record  ensures  that 
deficiencies  discovered  at  the  lower  levels  are  reported  to  the  highest  rated  level 
(AFFARS  Appendix  BB-3 04(e)). 

According  to  AFFARS  Appendix  BB-205(a),  an  offeror’s  proposal  shall  not  be 
compared  against  a  competitors.  Selected  “minimum  standards,”  determined  as  required 
to  meet  the  evaluation  criteria,  set  an  objective  standard  which  becomes  the  baseline  of 
measurement  on  how  weli  each  offeror’s  proposal  satisfies  the  evaluation  criteria.  The 
standards  may  be  qualitative,  quantitative,  or  both;  and  currently  do  not  appear  in  either 
the  Source  Selection  Plan  or  the  solicitation  (AFFARS  Appendix  BB-205(a)).  The 
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criteria  and  standards  selected  for  evaluation  are  directly  linked  to  the  areas  of 
performance  required  for  successful  completion  of  the  task,  are  not  so  restrictive  as  to 
discriminate  against  competing  offerors,  and  are  mutually  exclusive  of  other  criteria  to 
avoid  double  evaluating  an  offerors’  proposal  for  the  same  strengths  or  weaknesses 
(Cibinic  and  Nash,  1986). 

2.2.2. 3.2.2  Assessment  Criteria.  The  assessment  criteria 
serve  as  a  basis  for  evaluating  each  offeror’s  proposal  as  it  relates  to  the  relevant  , 
evaluation  criteria.  It  pertains  to  how  the  contractor  will  perform  the  effort  or  satisfy  the 
requirement.  Assessment  criteria  are  used  in  conjunction  with  evaluation  standards  to 
judge  how  well  an  offeror’s  proposal  satisfies  each  of  the  relevant  evaluation  criteria. 
Factors  and  subfactors  are  created  to  support  those  criteria.  Examples  of  common 
assessment  criteria  include  soundness  of  the  proposed  approach,  and  how  well  the  offeror 
understands  and  complies  with  the  requirement  (AFFARS  Appendix  BB-203 (b)(3)). 
Assessment  criteria  are  also  ranked  in  relative  order  of  importance  or  identified  as  of  equal 
importance  (AFFARS  Appendix  BB-204(c)(5)). 

2.2.2.3.23  Integrated  Evaluation  Matrix.  Table  5 
provides  an  example  of  an  Evaluation  Matrix  for  one  specific  Technical  Area.  This  table 
illustrates  an  example  of  the  relationship  between  specific  criteria,  assessment  criteria,  and 
the  color  and  risk  ratings  assigned  by  the  source  selection  evaluation  team. 

2.2.23.2.4  Cost  (Price)  Criterion.  The  cost  (price) 
criterion  relates  to  the  evaluation  of  the  offeror’s  proposed  costs  or  price.  Cost  (price)  is 
a  mandatory  evaluation  criterion  used  to  evaluate  every  AFFARS  Appendix  AA/BB 
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source  selection  for  realism,  completeness,  and  reasonableness.  Color  ratings  or  proposal 
risks  are  not  applied  to  the  evaluation  of  cost  (price)  (AFFARS  Appendix  BB-203(b)(l)). 

Table  5 

Evaluation  Matrix  for  Technical  Area  (Example) 

_ (AFFARS  Appendix  BB,  Attachment  BB-3) _ 

AREA:  TECHNICAL 


ITEM 

SUMMARY 

Green  (Low) 

Yellow  (Moderate)  Red  (High) 

ITEM 

ITEM 

ITEM 

SPECIFIC 

CRITERIA 

Reliability, 

Maintainability, 

and 

Producibility 

Systems 

Engineering 

and 

Integration 

Weapon 

Control 

Systems 

Design 

ASSESSMENT 

CRITERIA 

Soundness 

r* 

or 

Approach 

Green 

Yellow 

Red 

Understanding 

of 

Requirement 

Green 

Yellow 

Yellow 

Compliance 

with 

Requirement 

Green 

Green 

Green 

2.2.23.2.5  General  Considerations.  General 


considerations  relate  to  proposed  contractual  terms  and  conditions,  results  of  preaward 
surveys,  and  other  surveys  or  reviews  (AFFARS  Appendix  BB-203(c)).  Examples  of 
items  of  general  consideration  include  financial  capability  assessments,  production 
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readiness  reviews,  and  preaward  surveys.  General  considerations  are  not  evaluated  using 
color  coding,  but  are  considered  in  the  integrated  assessment  of  the  overall  proposal. 

2.2.23.2.6  Proposal  Risks.  Proposal  risks  are  assessments 
associated  with  schedule  and  performance  or  technical  aspects  of  the  program  (AFFARS 
Appendix  BB-203(c)).  Proposal  risks  are  not  evaluated  using  color  coding. 

2.2.23.2.7  Performance  Risks.  Performance  risks  relate 
to  cost  and  specific  criteria.  Performance  risk  is  assessed  for  each  area  or  as  an  overall 
general  assessment  of  performance  (AFFARS  Appendix  BB-203(e)).  Performance  risks 
are  not  evaluated  using  color  coding. 

23  Flow  of  Information  into  the  Source  Selection  Process 

Proposal  evaluation  is  an  assessment  of  the  proposal  and  the  offeror’s  ability  to 
perform  the  prospective  contract  successfully  (FAR  15.305(a)).  To  properly  evaluate 
each  proposal,  the  source  selection  officials  need  four  main  types  of  information:  technical, 
cost/price,  advisory,  and  contractor  past  performance.  A  technical  evaluation  is  an 
assessment  of  each  offeror’s  ability  to  accomplish  the  technical  requirements.  The 
technical  team  reviews  the  proposals,  and  based  upon  their  expert  knowledge,  assesses 
each  offeror’s  ability  to  successfully  perform  the  task  given  the  offeror’s  proposed 
technical  and/or  managerial  approach  to  solve  the  problem.  A  cost/price  evaluation  is  an 
assessment  of  the  proposed  cost/price  for  realism  and  reasonableness  in  relation  to  the 
technical  approach  proposed.  The  proposed  cost/price  is  also  compared  with  a 
Government  estimate  of  the  work  to  be  performed.  Expert  advisory  information  is  needed 
by  the  source  selection  team  in  areas  such  as  legal  advice;  contracting  and  business  policy 
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advice,  i.e.  latest  policy  issues,  pre-award  surveys,  equal  employment  opportunity 
compliance  reports,  and  small  business  subcontracting  efforts;  and  sometimes  additional 
expert  advice  is  required  in  technical,  cost,  and  past  performance  areas.  Past  performance 
information  is  another  indicator  of  an  offeror’s  ability  to  perform  the  contract  successfully. 
An  assessment  is  made  to  determine  the  risk  associated  with  the  contractor’s  past  work 
history  as  it  relates  to  the  current  acquisition.  Current  and  relevant  information  from 
reliable  sources  is  necessary  to  evaluate  the  general  trends  of  a  contractor’s  performance 
within  the  context  of  the  acquisition.  Figure  1  depicts  a  model  of  the  types  of  information 
required  by  the  SSAs  for  decision-making  during  a  source  selection. 

2.3. 1  Prior  Research  on  the  Automation  of  Information  into  the  Source 
Selection  Process. 

2.3. 1.1  Evaluation  Criteria  (Technical/Cost/Past  Performance).  In  a 
1991  AFIT  thesis,  Ken  Noffsinger  examined  the  evaluation  criteria  used  in  source 
selections  at  the  Air  Force  Logistics  Command.  Noffsinger  examined  contracting  files  at 
the  Wright-Patterson  Contracting  Center  and  Wamer-Robins  Air  Logistics  Center  to 
determine  the  types  of  evaluation  criteria  used  in  the  source  selection  process  and  to 
assess  their  usefulness  in  making  source  selection  decisions.  Thirty-two  source  selections 
awarded  between  the  years  of  1986  through  1991  were  evaluated  (Noffsinger,  1991:  41, 
47).  Noffsinger  developed  a  “frequency  count  database”  to  analyze  the  most  frequently 
used  evaluation  elements  identified  as  criteria  in  the  categories  of  General  Considerations, 
Assessment  Criteria,  and  Specific  Criteria  (Noffsinger,  1991:  48-50).  He  used  these 
elements  as  the  basis  for  analysis  of  their  relationship  to  the  award  decision.  He  organized 
the  data  into  twelve  categories:  cost  ranking,  overall  technical  color  rating,  overall 
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Figure  1.  A  Model  of  Information  Needed  to  Make  an  Optimal  Decision. 

technical  risk  rating,  overall  technical  color/risk  rating,  other  area  color  rating,  other  area 
risk  rating,  and  other  area  color/risk  rating,  past  performance  risk  rating,  cost  risk  rating, 
financial  risk  rating,  schedule  risk  rating,  and  contractual  risk  rating  (Noffsinger,  1991:  51- 
52).  He  recommended  continuation  of  this  research,  including  an  extension  into  “an 
examination  of  the  degree  to  which  evaluation  criteria  contribute  to  the  selection  of 
satisfactorily  performing  offerors,  which  substantially  meet  all  of  the  terms  and  conditions 
of  the  contract  for  the  duration  of  the  contract”  (Noffsinger,  1991 :  82).  He  suggested 
statistical  techniques,  such  as  discriminate  analysis  or  categorical  modeling,  to  identify 
evaluation  criteria  to  select  a  high  performing  contractor.  This  research  has  not  been 
accomplished.  To  collect  the  amount  of  contracting  information  required  to  obtain 
significant  results  for  this  research  would  be  difficult  because  contract  files  are  located  in 
various  locations,  and  manual  collection  through  requesting  copies  of  solicitations  and 
contractor  performance  data  would  be  cumbersome.  Done  manually,  this  research  would 
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be  almost  impossible  due  to  time  constraints  and  the  magnitude  of  information.  Use  of 
information  technology,  such  as  a  database  or  data  warehousing  system  for  data  collection 
and  storage,  and  the  inclusion  of  statistical  software  for  analysis,  make  this  continued 
study  feasible  and  its  potential  expansion  into  operational  capability  promising.  A  search 
to  obtain  the  information  stored  in  a  computer  would  permit  the  researcher  to  gather  data 
for  analysis.  With  increased  emphasis  in  awarding  to  contractors  who  have  a  good 
performance  record,  this  operational  capability  could  substantially  benefit  the  AF  with  the 
resulting  analysis. 

2.3. 1.2  Past  Performance.  In  a  1989  AFIT  thesis.  Major  Paul  Thurston 
attempted  to  “establish  the  requirements  baseline  and  conceptual  design  for  an  automated 
information  system,  based  upon  the  Contractor  Performance  Assessment  Reporting 
System  (CPARS),  in  order  to  collect,  process,  protect,  and  disseminate  contractor 
performance  assessments”  (Thurston,  1989:  1-8).  In  essence,  he  attempted  to  design  a 
database  system  which  could  be  used  during  source  selections  to  obtain  information 
regarding  contractor  performance  risk.  He  describes  in  detail  what  information  the 
database  should  include,  and  how  it  should  be  sorted  for  easy  manipulation  and  review  of 
the  data.  A  prototype  software  model  was  developed  but  was  never  implemented  for  use. 

2.3.1. 3  Expert  Advice.  In  another  1989  AFIT  thesis.  Major  Caisson 
Vickery  used  a  computer-aided  decision  support  system  (DSS)  in  the  source  selection 
process.  A  decision  support  system  (DSS)  is,  in  loose  terms,  an  “information  system” 
specifically  designed  to  assist  with  decision-making.  According  to  Gordon  and  Gordon,  a 
DSS  enhances  decision-making  by  providing  information  necessary  to  effectively  answer 
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complex  questions.  The  DSS  is  designed  to  evaluate  the  impact  of  alternative  decisions  in 
making  the  best  possible,  or  optimal,  choice.  The  system  is  designed  to  integrate  both 
internal  and  external  information  from  sources  for  incorporation  into  models  and 
analytical  tools  (Gordon  and  Gordon,  1995).  In  his  research,  Major  Vickery  used  a  DSS 
based  upon  an  analytical  hierarchy  process,  and  which  utilized  the  “Expert  Choice” 
software  program.  Vickery  stated  a  DSS  using  an  analytical  hierarchy  process  aligned 
real-world  decision-making  due  to  the  hierarchical  structure  of  the  source  selection  criteria 
into  area,  item,  and  factor  and  subfactor  levels.  Using  both  a  test  group  and  a  control 
group  for  experimentation,  Vickery  tested  the  DSS  for  effectiveness,  consistency,  speed, 
difficulty,  confidence,  and  understanding.  According  to  the  results  of  his  experiment,  the 
DSS  had  little  effect  on  effectiveness  and  consistency,  a  negative  impact  on  speed,  and  a 
positive  impact  on  ease,  confidence,  and  understanding  (Vickery,  1989).  The  negative 
impact  on  speed  could  have  been  due  to  participant  unfamiliarity  with  computers  or  the 
DSS.  “The  extent  to  decision  effectiveness  depends  on  the  user’s  familiarity  and  expertise 
with  the  decision  support  tool,  the  user’s  knowledge  about  the  problem  to  be  solved,  and 
the  interaction  of  the  cognitive  style  of  the  user  with  the  DSS”  (Gordon  and  Gordon, 
1995:331). 

2.4  Information  Technology  Overview 

According  to  Steve  and  Judith  Gordon,  in  Information  Systems,  A  Management 
Approach,  the  components  of  information  technology  consist  of  “computer  software, 
hardware,  database  management  systems,  and  data  communication  systems”  (Gordon  and 
Gordon,  1995:  13).  Gordon  and  Gordon  identify  four  types  of  information  systems: 
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automated  systems,  transaction  processing  systems,  management  information  systems,  and 
strategic  systems.  Automated  information  systems  (AIS)  are  those  systems  that  use 
information  technology  to  perform  tasks  that  would  otherwise  be  accomplished  manually. 
An  Expert  System  (ES)  is  a  type  of  AIS  automating  the  knowledge  of  expert  in  a 
particular  field.  A  transaction  processing  system  (TPS)  records  and  processes  an 
organization’s  routine  business  activities.  Management  information  systems  (MIS)  are 
designed  to  supply  the  information  an  organizational  manager  requires  to  function  better 
or  to  communicate  more  effectively.  MIS  systems  include  management  reporting  systems, 
decision  support  systems  (DSS),  and  executive  support  systems  (ESS)  or  executive 
information  systems  (EIS).  Strategic  systems  extend  a  systems  concept  beyond 
organizational  boundaries,  seeking  to  make  customers,  suppliers,  or  distributors  a  strategic 
partner  of  the  information  system  (Gordon  and  Gordon,  1995:  15-16).  “A  Database 
Management  system  (DBMS)  contains  software  comprising  of  programs  to  store,  retrieve, 
and  otherwise  manage  a  computerized  database  and  provides  interfaces  to  application 
programs  and  to  nonprogramming  users,  as  well  as  a  host  of  other  data  creation, 
manipulation,  and  security  features”  (Gordon  and  Gordon,  1995:  GL-6).  This  study 
concentrates  on  the  use  of  a  shared  DBMS  data  warehousing  system,  an  MIS  decision 
support  system,  and  an  AIS  expert  system. 

2.4.1  Data  Warehousing.  The  DoD  Standard  Procurement  System  (SPS)  is 
implementing  the  use  of  shared  data  warehouses  and  relational  databases  in  its  system’s 
integration  design.  Shared  data  warehouses  store  information  which  can  be 
simultaneously  accessed  by  personnel  at  various  locations.  The  SPS  will  contain  a 


database  to  store  audits  of  defense  contractors  conducted  by  the  Defense  Contract  Audit 
Agency  accessible  by  contract  specialists.  The  SPS  will  also  have  a  database  to  conduct 
price  analysis  using  historical  data  on  purchases  of  the  same  product  (DoD  SPS,  1998). 
Shared  data  warehouses  provide  the  basis  for  obtaining  data  to  conduct  statistical  analysis 
and  other  evaluations  useful  in  the  decision-making  process.  A  relational  database  model 
provides  a  logical  connection  between  data  objects,  or  entities,  and  internally  organizes  the 
data  in  a  way  that  makes  retrieval  of  information  quick  and  efficient.  The  use  of  a 
relational  database  management  system  supports  decision  support  and  ad  hoc  query 
applications  (Gordon  and  Gordon,  1995:  200-201). 

A  CPARs  database  has  recently  been  implemented  by  the  Headquarters  Air  Force 
Material  Command  (HQ  AFMC)  to  collect,  process,  and  store  contractor  past 
performance  information  within  HQ  AFMC  organizations.  The  development  of  this 
database  will  provide  past  performance  information  to  SSAs.  However,  this  database  is 
limited  in  scope  as  it  contains  only  CPARS  data  on  contracts  awarded  and  administered 
within  HQ  AFMC  organizations.  Expanding  the  use  of  information  technology,  such  as 
converting  to  a  data  warehousing  system  for  data  collection  and  storage,  could  provide 
more  Government  users  access  and  storage  of  information  regarding  the  performance  of 
contractors  whose  contracts  are  administered  by  other  AF  and  Government  agencies.  “A 
data  warehouse  supports  business  analysis  and  decision-making  by  creating  an  integrated 
database  of  consistent,  subject-oriented,  historical  information.  It  integrates  data  from 
multiple,  incompatible  systems  into  one  consolidated  database.  By  transforming  data  into 
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meaningful  information,  a  data  warehouse  allows  business  managers  to  perform  more 

substantive,  accurate  and  consistent  analysis”  (Prismsolutions,  1998). 

Significant  cost  benefits,  time  savings  and  productivity  gains  are  associated 
with  using  a  data  warehouse  for  information  processing.  First,  data  can  be 
easily  accessed  and  analyzed  without  time-consuming  manipulation  and 
processing.  Decisions  can  be  made  more  quickly  and  with  confidence  that 
the  data  is  both  timely  and  accurate.  Integrated  information  can  be  kept  in 
categories  that  are  meaningful  to  profitable  operation.  Trends  can  be 
analyzed  and  predicted  with  the  availability  of  historical  data.  [Data 
warehousing]  assures  that  everyone  is  using  the  same  data  at  the  same  level 
of  extraction,  which  eliminates  conflicting  analytical  results  and  arguments 
over  the  source  and  quality  of  data  used  for  analysis.  In  short,  the  data 
warehouse  enables  information  processing  to  be  done  in  a  credible,  efficient 
manner.  (Prismsolutions,  1998) 

Since  most  defense  contractors  contract  with  more  than  a  single  Government 
agency,  the  past  performance  information  included  in  an  all-encompassing  data  warehouse 
would  more  closely  reflect  the  contractor’s  complete  performance  position  and  provide 
the  SSA  with  a  more  accurate  assessment  of  the  contractor’s  ability  to  perform.  A  past 
performance  data  warehouse  system  could  be  maintained  at  a  central  location,  with 
interfaces  to  each  Government  agency,  to  provide  more  efficient  maintenance  and  security 
of  the  information  contained  within  the  data  warehouse.  The  inclusion  of  On-line 
Analytical  Processing  (OLAP)  statistical  or  analytical  software  within  the  system,  or 
through  the  use  of  an  interface  with  a  DSS  or  Expert  system,  could  provide  for  analysis  of 
performance  risk  based  upon  the  information  and  provide  SSAs  with  both  a  detailed  and 
summary  report  of  the  analysis.  Use  of  OLAP,  in  conjunction  with  a  data  warehouse, 
could  provide  SSAs  with  a  probability  factor  for  the  prediction  of  contractor  success  or 
failure  based  upon  its  current  and  relevant  past  performance  history  (Gordon  and  Gordon, 
1995). 
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2.4.2  Decision  Support  System.  A  DSS  is  specifically  designed  to  enhance 
decision-making  by  helping  managers  answer  complex  questions.  Most  DSSs  are 
designed  to  assist  with  strategic  or  managerial  level  questions,  however  a  well  designed 
DSS  system  can  be  used  at  any  level  (Laudon  and  Laudon,  1995).  A  DSS  provides 
information  required  for  effective  planning  and  organizing.  DSSs  are  designed  to  allow 
for  effective  decision-making  in  ambiguous  and  complex  environments.  They  provide 
managers  with  the  ability  to  quantitatively  analyze  alternative  choices  by  modeling  a 
complex  set  of  circumstances  which  the  decision  maker  can  manipulate.  By  manipulating 
the  various  parameters  of  the  model,  the  manager  can  assess  the  impact  of  diverse 
conditions  related  to  each  alternative  solution  to  make  a  better  decision  (Gordon  and 
Gordon,  1995:  330-331). 

According  to  Marakas  and  Elam,  a  user’s  decisions  can  be  influenced  by  a  DSS, 
and  software  which  is  designed  to  assist  decision-making  allows  more  creativity  in 
developing  possible  alternate  solutions  (Marakas  and  Elam,  1997).  A  strategically 
designed  DSS  could  be  useful  in  assisting  a  source  selection  team  in  the  development  and 
evaluation  of  the  source  selection  criteria  for  award  decisions.  A  DSS  could  assist  in  the 
planning  and  selection  of  the  specific  evaluation  criteria  required  by  evaluators  to  choose 
the  best  value  alternative,  and  also  during  the  evaluation  process,  by  allowing  them  to 
examine  possible  evaluation  scenarios  using  “what  if’  questions,  obtaining  expert  advice, 
and  searching  rules  and  regulations  for  conflicts  and  problems.  “Planned  systems  can 
handle  unforeseen  questions  by  providing  access  to  internal  and  external  data  and  models 
to  manipulate  the  data”  (Gordon  and  Gordon,  1995:  332). 
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2.4.2. 1  DSS  Components.  The  major  components  of  a  DSS  include  a 
database,  a  knowledge  base,  a  model  base,  and  a  user  interface.  The  database  provides 
access  to  internal  and  external  data  pertinent  to  decision-making.  “DSSs  use  historical 
data  from  the  database  to  form  the  baseline  that  mathematical  models  use  in  extrapolating 
from  past  to  present  to  future  conditions”  (Gordon  and  Gordon,  1995:  331).  A 
knowledge  base,  like  that  of  an  expert  system,  consists  of  rules  that  constrain  possible 
solutions  as  well  as  alternative  solutions  and  methods  for  evaluating  them.  The 
knowledge  base  provides  information  about  data  relationships  that  are  too  complex  for  a 
database  to  delineate.  A  model  base  includes  an  array  of  spreadsheets,  simulation 
packages,  forecasting  tools,  and  statistical  packages.  The  model  base  allows  the  user  to 
access  the  appropriate  tools  without  developing  a  new  model  each  time.  A  user  interface 
allows  the  user  to  control  which  data  and  models  to  include  in  the  analysis.  A  DSS  must 
be  designed  to  support  the  increased  flexibility  users  have  in  manipulating  data  and 
processing  information.  Since  DSSs  support  complex  decision-making,  users  generally 
analyze  many  alternatives  and  extensive  data  about  each  alternative.  A  DSS  should  be 
designed  to  support  the  manipulation  of  data  using  a  broad  range  of  graphical  and  tabular 
forms  (Gordon  and  Gordon,  1995:  331-332). 

2.4.2.2.  DSS  Benefits.  Benefits  of  the  DSS  include  improved  decision¬ 
making  through  a  better  understanding  of  the  business  situation,  an  increase  in  the  number 
of  decision  alternatives  that  can  be  examined,  the  ability  to  implement  ad  hoc  analysis,  and 
faster  response  to  expected  situations.  DSSs  also  improve  communication,  effective 
teamwork,  and  time  and  cost  savings  (Gordon  and  Gordon,  1995:  330-331). 
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2. 4. 2. 3  DSS  Business  Applications.  Wong  and  Monaco  conducted 
research  on  specific  business  applications,  to  determine  the  functional  focus  of  DSS  design 
in  planning  and  monitoring  systems.  The  focus  on  interpretation  and  prediction-type 
systems,  the  type  required  for  competitive  source  selection  decisions,  indicates  most  of  the 
design  and  development  is  centered  in  the  operations/production,  finance,  and  information 
systems  functions  (Wong  and  Monaco,  1995).  Little  research  has  been  focused  on  the 
design  and  development  of  a  DSS  system  specifically  for  competitive  procurements  which 
could  assist  the  decision-maker  in  determining  the  best  source.  Several  systems  have  been 
developed  to  assist  commercial  procurement  specialists  during  negotiations,  but  in 
Government  Contracting,  negotiations  occur  in  sole  source  procurements  rather  than 
competitive  acquisitions. 

Heng  Li  (1996)  used  a  combination  ES  and  DSS  database,  which  used  case-based 
reasoning  (CBR)  to  allow  negotiators  to  access  and  draw  on  actual  case  history  during  the 
negotiation  process.  Although  Heng  Li  doesn’t  specifically  acknowledge  the  system  as  a 
link  between  using  an  expert  system  (MEDIATOR)  and  a  DSS  database,  he  did  integrate 
the  technologies  to  providing  a  system  which  could  draw  on  past  experience  in  the  form  of 
a  case-based  reasoning  database  to  find  similarities  in  past  negotiations.  By  drawing  on 
historical  cases,  rather  than  expert  opinion,  this  integrated  system  acted  as  a  neutral,  third 
party  mediator  to  prevent  “deadlocks”  in  current  negotiations.  During  system 
development,  problems  were  encountered  in  collecting  previous  negotiation  cases  to 
include  in  the  database.  Direct  collection  was  difficult  because  negotiation  history  was 
seldom  recorded  and  documented.  Also,  determination  of  the  original  context  of  a 
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negotiation  was  often  difficult  to  establish  when  finding  similar  cases,  as  all  circumstances 
may  not  be  the  same  (Li,  1996).  One  draw-back  to  the  use  of  case-based  reasoning,  that 
is  not  discussed  by  Li,  is  the  use  of  historical  data  tends  to  set  a  precedence  for  similar 
decisions  even  though  circumstances  may  not  be  the  same  as  in  the  original  case,  or  even 
when  the  initial  decision  was  in  error. 

Rangaswamy  and  Shell  (1997)  used  a  “Negotiation  Support  System”  called 
NEGOTIATION  ASSISTANT  (NA)  designed  to  enable  negotiators  to  analyze  their  own 
preferences  and  to  provide  a  structured  negotiation  process  in  assisting  parties  in 
negotiating  optimal  trade-offs.  The  model  was  based  on  a  multiattribute  representation  of 
preferences  with  communications  occurring  over  a  computer  network.  The  parties  sent 
and  received  offers  and  counteroffers  via  the  system.  The  NA  assisted  negotiators  in 
reaching  a  mutual  agreement,  and  then  analyzed  the  final  agreement  to  see  if  either  party 
could  benefit  from  a  more  compromising,  Pareto-optimal  solution.  The  results  of  their 
experiment  suggest  parties  using  the  negotiation  support  system  in  structured  negotiation 
settings  achieved  better  outcomes  than  parties  negotiating  face-to-face  or  via  e-mail 
(Rangaswamy  and  Shell,  1997). 

Liberatore  and  Stylianou  (1994)  developed  a  system  for  the  design,  development, 
and  implementation  of  a  knowledge-based  decision  support  system  (KBDSS)  for  strategic 
market  assessment  which  could  possibly  be  modified  for  use  in  source  selection  decision¬ 
making.  The  resulting  system,  known  as  the  Strategic  Market  Assessment  System 
(SMAS)  integrated  a  “knowledge-based  system  with  scoring  models,  logic  tables,  and  an 
analytic  hierarchy  process  to  evaluate  whether  a  product  should  be  selected  for  full-scale 
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development”  (Liberatore  and  Stylianou,  1994).  The  analytical  hierarchy  process  (AHP) 
provides  a  more  structured  approach  for  determining  the  scores  and  weights  for  multi¬ 
criteria  scoring  models.  Multi-criteria  scoring  models  score  or  rate  each  alternative  in  a 
decision  problem  based  on  each  criterion  (Ragsdale,  1997:  716).  According  to  Liberatore 
and  Stylianou,  the  AHP  is  a  decision-making  method  for  prioritizing  alternatives  when 
multiple  criteria  must  be  considered.  It  allows  the  evaluator  to  structure  a  problem  into 
levels  such  as:  the  goal,  the  criteria,  the  ratings  scale,  and  the  projects  themselves.  The 
hierarchy  analyzes  the  impact  of  any  level  on  the  next  higher  level.  The  process  begins  by 
determining  the  relative  importance  of  the  criteria  in  meeting  the  goal  and  their  relative 
weights.  The  AHP-scoring  approach  forms  the  basis  of  a  decision  support  system  for 
project  evaluation.  A  knowledge-based  system,  like  ES,  encapsulate,  model,  and  process 
the  knowledge  of  experts.  In  conclusion  of  the  field  test,  Liberatore  and  Stylianou  state 
the  KBDSS  is  an  appropriate  modeling  framework  for  new  product  development 
(Liberatore  and  Stylianou,  1994). 

In  the  late  1980s,  the  U.  S.  Department  of  Housing  and  Urban  Development 
developed  a  decision  support  system  that  serves  headquarters  and  field  offices  for  the 
Urban  Development  Action  Grant  Program.  The  DSS  supports  the  operations  and  work 
processes  of  the  program  office  as  well  as  managerial  decision-making  regarding  grant 
applications.  The  DSS  includes  “quantitative  analysis  techniques  and  tools,  management 
decision-making  support,  ‘what-if  capability,  direct  and  intermediary  control  by  the 
decision  maker,  support  to  nonprogrammable  decisions,  real-time  analysis  capability,  and 
contingency  planning  features”  (Gordon  and  Gordon,  1995:  332). 
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2. 4. 2. 4  DSS  and  Time  Pressure  in  Source  Selections.  In  the  acquisition  field,  the  need 
to  process  large  amounts  of  information  in  a  relatively  short  time  has  an  impact  on 
decision  strategy  and  performance.  The  standard  time  for  completion  of  a  source 
selection  is  120  days.  The  goal  of  AF  Lightning  Bolt  #10  is  to  reduce  acquisition  cycle 
time  by  50%.  Cycle  time  is  measured  from  the  receipt  of  a  validated  user 
requirement/funding  commitment  to  contract  award  (SAF/AQXA,  1997).  If  a  difficult 
acquisition  requires  more  time  to  complete,  but  the  time  standard  is  inflexible,  the  shorter 
time  available  for  processing  would  increase  task  difficulty  for  the  evaluation  team.  The 
more  difficult  task  may  cause  the  selection  of  a  suboptimal  strategy  or  alternative,  which  in 
the  case  of  a  source  selection,  could  result  in  reduced  performance,  schedule  impacts,  and 
increased  costs.  According  to  Hwang  (1994),  a  DSS  can  be  used  to  counterbalance  the 
negative  impacts  time  pressure  has  on  decision  strategy  selection  and  performance.  'A 
theoretical  graph  of  performance  as  a  function  of  time  pressure  is  an  inverted  U-shape 
(Figure  2). 


Time  Pressure 

Figure  2.  U-Shaped  Effect  of  Time  Pressure  on 
Performance  (Hwang,  1994). 

Time  pressure  improves  decision  performance  to  a  point  after  which  additional 
time  pressure  decreases  performance.  “The  basic  assumption  of  cognitive  fit  is  that 


humans  have  limited  information  processing  capability  and  so  [they]  try  to  reduce  the  task 
complexity  in  solving  problems”  (Hwang,  1994:  201).  Information  systems  designed  to 
enhance  decision-making  have  an  important  role  in  supporting  decision-making  under  time 
pressure  (Hwang,  1 994).  These  information  systems  have  information  processing  capacity 
that  will  not  decrease  due  to  an  increase  in  task  difficulty  or  time  pressure  so  a  more 
extensive  analysis  of  alternative  solutions  is  feasible.  This  could  lead  to  the  selection  of 
better  solutions,  reduce  error  due  to  fatigue,  save  time  and  money,  and  prevent  schedule 
delays. 

2.4.3  Expert  Systems.  “Expert  systems  (ES)  are  computer  programs  designed  to 
make  available  to  users  the  knowledge  of  experts  in  a  particular  field”  (Dilworth,  1993: 
614).  In  his  1995  AFIT  thesis,  Shawn  Northrup  states  ES’s  have  emerged  as  one  of  the 
leading  mechanisms  for  providing...  optimal  answers,  and  have  become  one  of  the  user’s 
most  valued  tools”  (Northrup,  1995).  A  definition  by  Feigenbaum  states  “an  Expert 
System. .  .is  an  intelligent  computer  program  that  uses  knowledge  and  inference 
procedures  to  solve  problems  that  are  difficult  enough  to  require  significant  human 
expertise  for  their  solution”  (Northrup,  1995).  ES’s  generally  automate  the  role  of  an 
expert  in  a  given  field,  and  because  they  help  people  make  decisions,  they  are  tools  for 
decision  support  (Gordon  and  Gordon,  1995:  283). 

2.4.3. 1  Expert  System  Components.  The  basic  core  of  an  ES  contains  an 
inference  engine  and  a  knowledge  base,  or  database.  A  natural  language  interface  permits 
the  system  to  communicate  with  the  user.  The  system’s  knowledge  base  consists  of 
traditional  knowledge,  or  facts  and  other  information,  and  “if  then”  rules  to  determine  how 
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the  data  relate  to  other  data  and  to  potential  solutions.  These  inference  rules  are  rules  of 
good  judgment  and  reasoning  that  characterize  expert-level  decision-making  and  problem 
solving.  These  rules  are  established  by  interviewing  experts  in  the  field  to  determine  how 
they  make  decisions  (Dilworth,  1993:  614). 

The  inference  engine,  or  control  mechanism,  does  the  actual  problem  solving.  It 
contains  programs  that  allow  the  system  to  evaluate  the  rules  in  the  knowledge  base.  The 
inference  engine  determines  which  rules  to  invoke,  depending  upon  what  has  already  been 
determined  about  the  problem.  The  inference  engine  is  separate  from  the  knowledge  base, 
so  a  single  inference  engine  can  be  used  to  drive  several  knowledge  bases.  This  separation 
creates  expert  “shells”  which  can  accommodate  a  wide  range  of  disciplines,  depending  on 
the  data  loaded  on  each  knowledge  base  (Dilworth,  1993:  614).  ES  shells  use  an  interface 
engine  to  process  the  language  statements  and  data  supplied  by  users  to  reach  conclusions, 
answer  questions,  and  give  advice.  Figure  3  is  a  simple  illustration  of  a  short  program 
containing  two  facts  and  one  rule  which  may  be  expressed  in  the  language  of  an  expert 
system  shell.  An  inference  engine  would  use  this  program  to  determine  whether  or  not 
Mary  is  Alan’s  aunt  by  analyzing  and  comparing  the  “Rules”  knowledge  base  with  the 
“Facts”  knowledge  base  to  provide  an  answer  (Gordon  and  Gordon,  1995:  134). 

2.4.3. 2  Expert  Systems  and  Artificial  Intelligence.  The  knowledge  base 
is  the  heart  of  the  expert  system.  The  first  knowledge  bases  used  “if-then”  type  rules  but 
these  rules  became  inefficient  because  the  knowledge  bases  grew  so  enormous  and 
complex  it  made  inferencing  impossible.  Later,  an  approach  called  Bayesian  Knowledge 
Base  (BKB)  was  used  to  overcome  incompleteness  and  provide  more  flexibility  as  it  can 
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Figure  3.  An  Illustration  of  Rules  and  Facts  for  an  Expert  System 
(Gordon  and  Gordon,  1995: 134). 

represent  a  type  of  cyclic  knowledge  and  can  inference  using  incomplete  probability  tables 
(Northrup,  1995).  More  recent  systems  operate  using  Artificial  Intelligence  (AI).  Alisa 
“branch  of  computer  science  that  attempts  to  give  computers  the  ability  to  understand 
language,  to  solve  problems  that  call  for  reasoning,  and  to  learn — in  sum,  to  emulate 
human  methods  of  learning  and  solving  problems”  (Dilworth,  1993:  613).  ES’s  are  the 
most  widely  used  operation  to  employ  AI.  AI  approximates  human  reasoning  through  the 
manipulation  of  symbols  (i.e.,  Rule  Base  in  Figure  2),  as  opposed  to  conventional  systems 
which  manipulate  numbers  and  use  algorithms  to  solve  problems.  Conventional  systems 
process  data  and  make  mathematical  calculations  quicker  than  the  human  brain,  and  they 
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do  not  become  fatigued  by  repetitious  tasks.  For  this  reason,  conventional  computers  can 
be  considered  better  than  humans  at  many  tasks.  However,  for  tasks  that  require 
creativity  or  new  rules  and  insights,  conventional  systems  may  not  be  as  successful  as 
humans.  The  ability  to  use  incomplete  data  for  analysis  makes  AI  systems  more  flexible 
than  conventional  systems.  AI  systems  “make  assumptions,”  and  simulate  inductive 
reasoning  by  comparing  incomplete  data  to  idea  models”  (Dilworth,  1993:  613).  Systems 
that  use  both  simulated  inductive  reasoning  and  conventional  standards  with  set  algorithms 
resemble  the  intuitive  and  logical  thought  processes  that  humans  use  to  solve  problems. 

AI  systems  are  good  at  orderly  thought.  With  the  use  of  artificial  intelligence,  expert 
systems  can  offer  advice.  Expert  systems  can  also  act  as  interpreters  to  enhance 
communications  between  various  automated  systems  (Dilworth,  1993:  613-614). 
Integration  of  an  expert  system  with  one  or  more  components  of  a  DSS  allows  the  ES  to 
act  as  a  consultant  to  the  DSS.  Stand-alone  ES’s  are  limited  in  their  strategic  decision¬ 
making  ability.  Integration  of  an  ES  into  a  decision  support  system  makes  the  decision 
maker  better  equipped  to  handle  broad,  ad  hoc,  or  unique  problems  than  if  using  a  stand¬ 
alone  expert  system.  Using  a  system  based  on  ES/DSS  integration  could  improve  a 
decision-maker’s  performance  in  a  strategic  planning  environment  by  helping  identify 
problems  and  providing  relevant  theoretical  models  (Wong,  1995). 

2.4.4  Microsoft  Excel  SOLVER  as  Model  Base.  The  Air  Force  competitive 
source  selection  process  is  based  upon  a  hierarchical  evaluation  system  (paragraph 
2. 2.2. 3.2)  to  determine  award  of  the  proposed  alternative  that  represents  the  best  value  to 
the  government.  An  alternative  is  “a  course  of  action  intended  to  solve  a  problem” 
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(Ragsdale,  1997:  671).  Air  Force  source  selections  are  based  on  factors  other  than  price 
alone  to  determine  who  receives  contract  award.  Due  to  this,  both  quantitative  and 
qualitative  data  flows  into  the  source  selection  process  and  must  be  evaluated  and 
analyzed  in  comparison  to  the  evaluation  criteria  and  standards  provided  in  the  source 
selection  plan  for  each  individual  acquisition.  A  DSS  model  base  can  assist  the  source 
selection  team  in  their  manipulation  and  analysis  of  the  information  flowing  into  the 
decision  process. 

“Management  Science  [(or  Operations  Research)]  is  the  field  of  study  which  uses 
computers,  statistics,  and  mathematics  to  solve  business  problems”  (Ragsdale,  1997:  1). 
As  stated  in  paragraph  2.4.2. 1,  a  model  base  includes  an  array  of  spreadsheets,  simulation 
packages,  forecasting  tools,  and  statistical  packages.  The  model  base  allows  the  user  to 
access  the  appropriate  tools  without  developing  a  new  model  each  time.  The  Microsoft 
Excel  program  is  a  management  science  tool  which  has  the  capability  to  perform 
mathematical  optimization  using  linear,  integer  linear,  and  nonlinear  programming 
techniques.  It  also  includes  goal  programming  and  multiple  objective  optimization 
features.  It  has  the  capacity  to  perform  simple  and  multiple  regression  analysis, 
discriminant  analysis,  and  time  series  analysis.  It  can  analyze  risk  through  the  use  of 
simulation  techniques.  It  also  features  queuing  theory,  project  management,  and  decision 
analysis  techniques  (Ragsdale,  1997).  These  features  can  be  used  in  conjunction  with  the 
graphics,  charts,  and  other  features  contained  in  the  Windows  95  or  Window  97  software 
packages  (Excel  spreadsheet  programs,  Access  database,  Powerpoint  graphics.  Word  for 
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text)  to  analyze  and  present  a  consolidated  evaluation  briefing  to  the  SSA  for  final 


decision  selection. 
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Figure  4.  Line  Graph  Evaluation  Comparison  of  Normal  vs.  Deviated  Bids. 

One  example  of  using  an  Excel  spreadsheet,  in  conjunction  with  graphics,  which 


could  assist  the  SSA  to  visually  examine  the  existence  of  offeror  front  end  loading  or 


unbalanced  bids  is  through  the  use  of  line  graphs.  The  SSA  could  graphically  compare 
each  offeror’s  line  item  costs  to  the  Government  estimate  and  look  for  visual  deviations  of 


extremely  high  or  low  proposed  costs  per  line  item  (Figure  4).  As  can  be  seen  in  Figure  4, 
under  normal  bidding,  all  offerors  proposed  costs  and  the  Government  estimate  are 
comparable  with  only  minor  deviations  between  proposed  costs  and  the  Government 
estimate  (assuming  the  Government  estimate  is  a  good  estimate).  An  offeror  may 
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sometimes  see  an  opportunity  to  front  load  the  costs  of  the  items  to  be  delivered  first,  in 
order  to  be  paid  in  advance  of  incurring  the  costs  of  the  latter  delivered  items.  In  Figure 
4,  Offeror  1  has  front  loaded  the  costs  of  contract  line  item  numbers  (CLINs)  0001  and 
0002  and  reduced  the  costs  of  the  CLIN  to  be  delivered  at  a  later  date.  If  the  award  was 
to  be  made  to  the  lowest  priced  bid,  evaluation  of  total  price  alone  would  have  awarded 
the  contract  to  Offeror  1.  If  CLINs  0003  and  0004  were  Option  CLINs,  which  may  or 
may  not  be  exercised,  Offeror  1  may  not  be  the  lowest  priced  offeror,  even  though  his 
offered  price  is  the  lowest  total  price  proposed.  In  unbalanced  bidding,  Offeror  1  has 
proposed  an  extremely  low  cost  for  CLIN  0001  but  an  extremely  high  cost  for  CLIN 
0003.  Unbalanced  bidding  is  usually  due  to  a  perceived  Government  emphasis  on  specific 
evaluation  criteria  in  the  solicitation,  or  on  a  perceived  quantity  of  items  the  offeror 
estimates  will  be  desired  by  the  Government  per  specific  CLIN.  This  normally  happens  in 
an  Indefinite  Delivery,  Indefinite  Quantity  (IDIQ)  or  Requirements  type  contract,  where 
the  offeror  attempts  to  lower  his  costs  on  items  which  he  estimates  the  Government 
desires,  and  raises  the  price  on  the  items  he  estimates  the  Government  requires.  Manual 
evaluation  of  proposals  for  front  end  loading  or  unbalanced  bids  is  tedious  and  time- 
consuming  for  the  contracting  official,  especially  as  the  number  of  offerors  and  the  number 
of  contract  line  items  increase.  By  placing  the  offerors  proposed  prices  in  a  spreadsheet, 
and  graphically  displaying  the  costs  per  CLIN,  the  contracting  official  need  only  look  for 
abnormalities  that  deviate  from  the  Government  estimate  and  competing  offerors  proposed 
prices.  Once  the  evaluation  team  is  properly  trained  in  using  these  techniques,  evaluation 
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time  should  be  reduced  as  the  computer  can  strategically  manipulate  the  data  and  analyze 
it  faster,  with  more  accuracy  than  manual  calculations. 

2. 4. 4.1  The  Management  Science  Approach  to  Decision-Making. 
Management  science  (or  operations  research)  “involves  applying  the  methods  and  tools  of 
science  to  management  and  decision-making”  (Ragsdale,  1997:  1).  In  the  past, 
management  science  was  a  specialized  field  which  required  practitioners  to  have  a 
mainframe  computer  and  an  advanced  knowledge  of  mathematics  and  computer 
programming  languages  to  manipulate  data  (Ragsdale,  1997:  3).  The  concept  of  using 
management  science  models  in  problem  solving  and  decision  analysis  is  not  new.  Both 
Vickery  (1989)  and  Noffsinger  (1991)  discuss  the  use  of  the  Analytical  Hierarchical 
Process  (AHP)  as  a  potential  model  for  decision-making  in  the  Government  source 
selection  process.  Vickery’s  study  on  the  use  of  Decision  Support  Systems  suggests  the 
use  of  an  AHP  approach  developed  by  Saaty  in  1982,  as  the  proposed  method  of  ranking 
in  source  selection  decision-making  (Vickery,  1989).  Noffsinger’ s  study  on  the 
relationship  evaluation  criteria  has  on  the  award  decision  discusses  commercial  purchasing 
practices  implementing  AHP  as  the  “method  by  which  purchasing  managers  may  attack 
the  supplier  selection  problem”  (Noffsinger,  1991:  35).  However,  the  costs  and 
complexity  of  past  technology  limited  the  use  of  management  science  tools  in  the  business 
world.  The  emergence  of  personal  computers  (PCs)  and  the  development  of  easy-to-use 
spreadsheets  have  made  these  tools  more  practical  and  available  for  use  in  business 
decision-making  (Ragsdale,  1997:  3).  Use  of  these  tools  in  the  design  and  development  of 
Information  Systems  for  business  applications  and  decision-making  is  a  means  to  change  a 
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“tactical”  information  systems  design  into  a  more  “strategic”  system  design.  A  statement 
from  CIO,  Mr.  Ron  J.  Ponder,  at  Sprint  Corporation  (and  former  CIO  at  Federal  Express 
Corp.)  emphasizes: 

If  IS  [information  system]  departments  had  more  participation 
from  operations  research  analysts,  they  would  be  building  much 
better,  richer  IS  solutions. . .  Ponders  and  others  say  analyst  trained 
in  operations  research  or  management  science  can  turn  ordinary 
information  systems  into  money-saving  decision-support  systems 
and  are  ideally  suited  to  be  members  of  the  business  process  re¬ 
engineering  team  (Ragsdale,  1997:  11). 

2. 4. 4. 2  Mathematical  Programming  or  Optimization.  Mathematical 
programming  (MP),  or  optimization,  is  “a  field  of  management  science  that  finds  the 
optimal,  or  most  efficient,  way  of  using  limited  resources  to  achieve  the  objectives  of  an 
individual  or  business”  (Ragsdale,  1997:  16).  Some  business  applications  implementing 
MP  techniques  in  decision-making  include  the  determination  of  product  mix  to  maximize 
profits  or  to  satisfy  demand  at  a  minimum  cost,  the  routing  and  logistics  of  merchandise  to 
stores  to  minimize  transfer  costs,  and  in  financial  planning  the  minimization  of  the  amount 
of  taxes  paid  while  staying  within  the  constraints  of  the  tax  laws  (Ragsdale,  1997:  17). 

Vickery  discusses  optimization  as  making  “the  best  possible  decision  under  the 
circumstances  at  hand”  (Vickery,  1989:  1 1).  His  research  studied  various  opinions  on 
optimizing,  satisficing,  and  decision-making,  including  Simon,  1976;  Janis  and  Mann, 

1977;  Lohaus,  1985;  and  Cook,  1987.  Simon’s  approach  for  finding  the  optimal  solution 
was  like  a  rational  man  who  makes  decisions  based  upon  perfect  knowledge  of  ail 
alternatives,  and  who  examines  the  alternatives  using  an  economic  cost-benefit  analysis. 
Lohaus’ s  approach  was  like  a  decision-maker  who  “considers  all  possible  solutions  by 
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weighing  the  alternatives  against  each  other  until  the  best  or  optima!  solution  is  found.” 
Lohaus  further  describes  the  process  of  “satisficing,”  rather  than  “optimizing,”  to  find  a 
solution  that  is  “good  enough.”  Satisficing  involves  identifying,  testing,  and  gaining 
acceptance  of  an  “acceptable”  solution  due  to  the  human  minds  limitations  to  optimize. 
Janis  and  Mann  had  previously  categorized  obstacles  to  optimization  as  limitations  of  the 
mind  or  bureaucratic  obstacles.  Satisficing  is  evident  in  Cook’s  research.  Cook  found 
that  “only  45.5%  of  managers  optimize”  (Vickery,  1989:  1 1-14).  Vickery’s  research 
demonstrates  the  need  to  use  information  technology  when  examining  alternatives  to  assist 
the  “human  mind”  to  optimize  in  decision-making  rather  than  to  select  an  acceptable 
solution  and  satisficing.  An  information  system  with  the  proper  decision-making  tools 
can  analyze  iterations  of  information  without  interference  from  human  constraints,  such  as 
tediousness,  fatigue,  or  boredom. 

2. 4.4. 3  Optimization  Tools.  An  optimization  model  can  be  written  using 
several  different  tools:  a  spreadsheet,  a  modeling  language,  or  a  solver  library  for  custom 
programs.  The  spreadsheet  is  the  basic  optimization  tool.  Modeling  languages,  such  as 
algebraic  modeling  languages,  are  written  at  a  higher  level  language  than  spreadsheet 
tools.  These  modeling  languages  are  designed  to  be  more  flexible  than  spreadsheets  as  the 
amount  of  data  used  in  the  model,  or  dimensions,  begin  to  change.  Solver  libraries  for 
customs  programs  are  created  to  increase  efficiency  of  a  model  or  specific  solver  by 
capitalizing  on  special  features.  These  libraries  provide  special  translation  codes  to  allow 
the  solver  to  run  numerous  variations  of  an  optimization  model  simultaneously  and  at  a 
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faster  speed  (Compass,  1998).  A  system  using  the  solver  library  translation  codes  could 
permit  evaluations  of  award  by  several  contracting  officials  simultaneously. 

The  spreadsheet  approach  was  chosen  for  analysis  in  this  study  due  to  its 
commercial  availability  and  easy  accessibility  to  Government  contracting  and  source 
selection  personnel.  Spreadsheets,  such  as  Microsoft  Excel,  are  becoming  increasingly 
important  for  “prototyping  and  building  mathematical  programming-based  decision- 
support  models”  (Compass,  1998).  Spreadsheets  can  be  used  to  create,  modify,  format, 
and  audit  the  model,  and  as  demonstrated  in  paragraph  2.4.4,  it  is  easy  to  add  charts  and 
graphics  for  presentations.  Spreadsheets  are  “inherently  ffee-form  and  impose  no 
particular  structure  on  the  way  we  model  problems”  (Ragsdale,  1 997).  Once  personnel 
become  accustomed  to  Excel  spreadsheet  modeling,  the  learning  curve  for  model 
development  is  reduced.  In  addition,  communication  of  results  to  management  and  other 
personnel  is  facilitated  because  of  a  general  familiarity  of  spreadsheet  programs  (Compass, 
1998).  “Microsoft  Excel’s  SOLVER  provides  one  of  the  most  effective  ways  to 
manipulate  LP  models  in  a  spreadsheet”  (Ragsdale,  1997:  109). 

2. 4.4. 4  Optimization  Models  Expressed  Mathematically .  An 
optimization  model  generates  a  “best  solution”  based  on  a  set  of  constraints  (Vickery, 
1989:  16).  The  three  elements  of  an  optimization  problem  are  decisions,  constraints  (or 
restrictions),  and  an  objective  (or  goal)  (Ragsdale,  1997:  17-18).  Optimization  problems 
can  be  expressed  mathematically  using  these  three  elements.  Decisions,  or  decision 
variables,  are  represented  by  symbols  such  as  Xi,  X2, . . .  ,  X„.  The  constraints  can  be 
represented  in  a  variety  of  forms.  Three  general  ways  of  stating  possible  constraints  are: 
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A  less  than  or  equal  to  constraint:  / (Xi,  X2,  ...  ,  Xn)  <  b 
A  greater  than  or  equal  to  constraint:  / (Xi,  X2,  ...  ,  X„)  >  b 
An  equal  to  constraint:  f  (Xi,  X2,  ...  ,  Xn)  =  b 
(Ragsdale,  1997:  19) 

where/ (Xh  X2,  . . .  ,  Xn)  is  referred  to  as  the  left-hand  side  (LHS)  of  the  constraint,  and  b 

is  referred  to  as  the  right-hand  side  (RHS)  or  resource  limitation  of  the  constraint 

(Ragsdale,  1997:  19).  The  objective,  or  goal,  is  represented  by  an  objective  function  in 

the  general  format: 

MAX  (or  MIN):  f(Xh  X2,  ...  ,  Xn) 

(Ragsdale,  1997:  19) 

The  objective  function  describes  some  function  of  the  decision  variables  that  the  decision¬ 
maker  wants  to  minimize  or  maximize.  “The  goal  in  optimization  is  to  find  the  values  of 
the  decision  variables  that  maximize  (or  minimize)  the  objective  function  without  violating 
any  of  the  constraints”  (Ragsdale,  1997:  19).  The  mathematical  formulation  of  an 
optimization  problem  in  general  format  is  represented  as: 

MAX  (or  MIN):  f0  (Xu  X2,  ,  Xn) 

Subject  to:  f\  (Xi,  X2,  ...  ,  Xn)  <  bi 
/(XlsX2,  ...,Xn)>b* 

/n(X,,X2,  ...,Xn)  =  bm 
(Ragsdale,  1997:  19) 

Functions  of  MP  problems  can  be  expressed  in  linear  or  nonlinear  form,  and  the  optimal 
values  of  the  decision  variables  can  be  expressed  in  integer  or  fractional  values.  Many 
techniques  have  been  developed  to  solve  various  types  of  MP  problems.  Linear 
Programming  (LP)  is  the  basis  of  many  of  these  techniques  (Ragsdale,  1997:  20). 

2.4.5  Linear  Programming  and  Microsoft  Excel  SOLVER.  Linear 
Programming  (LP)  involves  MP  problems  with  linear  objective  functions  and  linear 
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constraints  (Ragsdale,  1997:  20).  LP  solvers,  such  as  Microsoft  Excel’s  SOLVER, 
require  the  model  to  be  specified  by  a  matrix  of  coefficients  (with  rows  for  constraints  and 
columns  for  variables)  (Compass,  1998).  The  constraints  of  an  LP  model  outline  the  set 
of  feasible  solutions,  or  feasible  region,  for  a  problem.  The  feasible  region  is  “the  set  of 
points  or  values  that  the  decision  variables  can  assume  and  simultaneously  satisfy  all  the 
constraints  in  the  problem”  (Ragsdale,  1997:  24).  The  LP  model  uses  the  data  within  the 
feasible  region  to  determine  the  optimal  solution,  or  solutions  that  minimizes  or 
maximizes  the  value  of  the  objective  function.  An  LP  problem  is  considered  infeasible 
when  there  is  no  way  to  simultaneously  satisfy  all  of  the  constraints  in  the  problem 
(Ragsdale,  1997:  33,  37). 

An  LP  problem  can  occasionally  have  more  than  one  optimal  or  alternative 
solution.  This  occurs  when  there  is  more  than  one  possible  outcome  or  point  in  the 
feasible  region  that  maximizes  or  minimizes  the  value  of  the  objective  function  or  end 
goal.  Alternate  solutions  indicate  there  are  two  or  more  ways  to  meet  the  same  goal  (to 
maximize/minimize  the  objective).  Alternative  solutions  are  anomalies  which  sometimes 
occur  so  they  must  be  considered  in  the  final  decision.  The  Microsoft  Excel  SOLVER 
program  can  provide  a  sensitivity  analysis  report  for  the  problem  to  assist  the  decision 
maker  identify  the  existence  of  alternative  solutions  so  they  may  be  examined  for  selection 
preference  (Ragsdale  1997). 

Many  variations  of  LP  problems  can  be  formulated  using  the  SOLVER  program 
through  the  use  of  various  constraints.  An  integer  linear  program  restricts  the  values  of 
the  decision  variables  to  assume  only  integer  values.  Appendix  B  contains  a  sample 
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integer  linear  problem  for  a  contract  award  problem.  The  objective  of  the  sample 
problem  is  to  minimize  total  costs  within  specific  constraints  proposed  by  each  offeror. 
The  use  of  the  SOLVER  program  allows  the  purchasing  agent  to  determine  how  much 
cement  to  purchase  from  each  offeror  and  minimize  the  total  costs.  The  decision 
variables,  or  changing  cells,  in  the  SOLVER  program  are  stated  in  integer  form  due  to  the 
integer  constraints  within  the  problem.  The  ability  to  constrain  or  limit  certain  variables  to 
assume  only  integer  form  allows  the  model  to  express  constraints  for  quantity  discounts 
and  minimum  purchase  order/purchase  size  more  accurately  by  adding  binary  and  linking 
variables.  Binary  variables,  Y„  can  assume  only  two  integer  values:  0  or  1.  Binary 
variables  are  useful  for  depicting  and  enforcing  relationships  among  decision  variables 
(Ragsdale,  1997). 

The  sample  contracting  problem  in  Appendix  B  demonstrates  how  the  use  of  basic 
LP  and  integer  LP  tools  provide  the  means  to  formulate  a  generic  model  for  use  in 
determining  the  optimal  solution  in  sealed  bidding  acquisitions.  The  basis  for  award  in 
sealed  bidding  is  to  evaluate  the  offers  and  award  to  the  responsible  offeror  whose  bid  is 
the  most  advantageous  to  the  Government,  considering  only  price  and  the  price-related 
factors  (FAR  14. 101(d)(e)).  The  award  goes  to  the  overall,  lowest  priced  offeror  whose 
bid  conforms  to  the  requirements  of  the  invitation  for  bids  (IFB).  Binary  variables  permit 
the  addition  of  constraints  for  price  adjustment  clauses.  These  price  adjustment  clauses 
(or  price-related  factors)  add  constraints  for  factors  such  as  discounts,  economic  price 
adjustments,  transportation  costs,  life  cycle  costs,  taxes,  options,  or  Buy  American  Act 
provisions  which  must  be  considered,  in  conjunction  to  the  total  proposed  price,  to 


determine  the  most  advantageous  award.  Bids  can  be  examined  for  unbalanced  bids  or 
front  end  loading  through  the  use  of  Excel’s  graphics  programs.  Constraints  can  also  be 
added  to  allow  award  on  an  “all-or-none”  basis  or  to  divide  the  acquisition  into  multiple 
awards.  If  multiple  awards  are  considered,  a  constraint  to  add  the  $500  administrative  fee 
can  be  linked  to  each  award  using  binary  variables  to  determine  the  individual  award  or 
combination  of  items  that  result  in  the  lowest  aggregate  cost  to  the  Government,  including 
the  assumed  administrative  fees.  These  optimization  methods  will  allow  the  contracting 
official  to  evaluate  all  proposals  to  determine  the  most  advantageous  award  alternative  for 
the  Government  and  to  complete  the  task  within  the  bid  evaluation  time. 

2. 4 . 5. 1  Goal  Programming  and  Multiple  Objective  Optimization. 
Additional  variations  to  LP  programs  include  Goal  Programming  (GP)  and  Multiple 
Objective  Linear  Programming  (MOLP)  techniques.  Goal  Programming  provides  a  way 
of  analyzing  solutions  to  decision  problems  which  can  be  stated  as  goals  with  target 
values.  These  goals  are  restated  as  constraints  using  deviational  variables,  which  measure 
the  amount  of  deviation  a  given  solution  varies  from  a  particular  goal.  The  objective  of  a 
GP  problem  is  to  minimize  some  weighted  function  of  the  deviational  variable.  The 
weights  of  the  deviational  variables  can  be  manipulated  to  analyze  alternate  solutions. 

This  feature  allows  the  analysis  of  trade-offs  within  problem  constraints.  A  minor 
alteration  which  can  be  implemented  for  use  with  GP  problems  is  the  MINIMAX 
objective.  This  alteration  changes  the  objective  function  of  the  problem  to  allow  the 
program  to  minimize  the  maximum  deviation  of  any  particular  goal  (Ragsdale,  1997). 
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MOLP  allows  for  the  inclusion  and  analysis  of  secondary  objective  functions 
within  a  single  problem.  For  example,  one  objective  might  be  to  maximize  the  technical 
score  while  the  secondary  objective  might  be  to  minimum  costs.  Problems  with  multiple 
objectives  require  analyzing  trade-offs  between  the  different  objectives  using  the 
MINIMAX  component  (Ragsdale,  1997:  276). 

GP  and  MOLP  have  potential  for  use  in  evaluating  FAR  Part  15  and  AFFARS 
Appendix  AA/BB  competitive  acquisitions.  The  ability  to  add  weights  permits  trade-off 
analysis,  the  MINIMAX  component  permits  evaluation  of  multiple  objectives,  such  as 
minimize  costs  and  maximize  technical  score,  and  the  ability  to  select  the  alternative  with 
the  minimize  deviation  from  the  target  permits  a  best  value  decision. 

2.4. 6  Decision  Analysis.  Two  primary  causes  that  make  decision-making  a 
difficult  task  include  uncertainty  of  the  future  and  conflicting  values  or  objectives 
(Ragsdale,  1997).  The  goal  of  decision  analysis  is  to  assist  in  making  good  decisions. 
Using  a  structured  approach  to  decision-making  enhances  understanding  and  perception 
about  the  problems  faced.  Decisions  require  at  least  two  alternatives  to  evaluate  or  solve 
a  problem.  An  alternative  is  “a  course  of  action  intended  to  solve  a  problem”  (Ragsdale, 
1997:  671).  Alternatives  are  evaluated  on  the  value  they  attribute  to  one  or  more  decision 
criteria.  The  criteria  in  a  decision  problem  represent  various  factors  important  to  the 
decision  maker  which  are  influenced  by  the  alternatives.  One  problem  in  comparing 
alternatives  is  that  not  all  criteria  can  expressed  in  monetary  or  quantitative  values.  Also, 
the  values  assigned  to  the  various  decision  criteria  under  each  alternative  are  dependent 
upon  numerous  states  of  nature  which  could  potentially  occur.  A  state  of  nature  is  “a 


62 


future  event  that  is  not  under  the  control  of  the  decision  maker”  (Ragsdale,  1997:  670- 
671).  In  decision  analysis,  payoff  matrices  and  decision  trees  are  two  tools  which  allow 
the  decision  maker  to  analyze  alternatives  given  a  particular  state  of  nature.  A  payoff 
matrix  is  “a  table  that  summarizes  the  final  outcome  (or  payoff)  for  each  decision 
alternative  under  each  state  of  nature”  (Ragsdale,  1997:  672).  A  decision  tree  is  a 
graphical  form  of  a  payoff  matrix.  To  construct  a  payoff  table  or  decision  tree,  each 
alternative  and  each  possible  state  of  nature  must  be  identified  (Ragsdale,  1997). 
Identification  of  all  possible  alternative  and  state  of  nature  is  not  always  possible  in  the 
decision-making  process  of  Government  source  selections.  In  addition,  the  decision 
makers  in  a  Government  source  selection  often  use  more  than  one  criterion  or  objective  to 
evaluate  alternatives,  and  sometimes  these  criteria  conflict  with  one  another.  For  example. 
Government  SSAs  want  to  reduce  performance  risk  and  cost,  but  simultaneously  desire  to 
maximize  the  technical  capability.  Trade-offs  must  be  assessed  to  identify  the  decision 
that  achieves  the  most  satisfying  balance  between  the  opposing  criteria.  Multi-criteria 
Decision-making  techniques,  such  as  the  Multi-criteria  Scoring  Model  and  Analytical 
Hierarchy  Process,  allow  flexibility  to  assess  trade-offs  in  selection  of  alternatives 
(Ragsdale,  1997:  670-715). 

2.4.6.1  The  Multi-Criteria  Scoring  Model  The  multi-criteria  scoring 
model  is  “a  simple  procedure  [to]  score  (or  rate)  each  alternative  in  a  decision  problem 
based  on  each  criterion”  (Ragsdale,  1997:  716).  The  score  for  alternative^'  on  criterion  i 
is  identified  by  sy  and  weights  are  identified  by  w,.  In  a  scoring  model,  each  decision 
alternative  is  assigned  a  value  between  0  and  1  to  reflect  its  relative  value  to  the  decision 
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maker.  Weights  are  subjectively  assigned  to  each  criterion  based  upon  its  relative 
importance  to  the  decision-maker.  A  weighted  average  score  is  calculated  for  each 
alternative  as: 

Weighted  average  score  for  alternative  j  =  ^w,% 

(Ragsdale,  1997:  716) 

The  alternative  with  the  highest  weighted  average  score  is  selected  as  the  best  value 
alternative  (Ragsdale,  1997:  716).  This  technique  is  similar  to  some  of  the  scoring 
methods  used  in  FAR  Part  15  competitive  acquisitions  for  calculating  a  best  value  award. 
This  method  has  potential  for  use  in  AFFARS  Appendix  AA  and  BB  source  selections, 
with  minor  modifications  to  convert  the  color  coding  into  a  standardized  number  system. 

2.4. 6.2  Analytical  Hierarchy  Process.  An  enhancement  to  the  multi¬ 
criteria  scoring  model  is  the  analytical  hierarchy  process  (AHP).  AHP  is  used  when  the 
decision  maker  has  problems  subjectively  determining  the  criterion  scores  and  weights.  As 
stated  in  paragraph  2.4.2.3,  the  AHP  provides  a  more  structured  approach  for  determining 
the  scores  and  weights  for  multi-criteria  scoring  models  (Ragsdale,  1997:  717).  The  AHP 
is  a  decision-making  method  for  prioritizing  alternatives  when  multiple  criteria  must  be 
considered.  It  allows  the  evaluator  to  structure  a  problem  into  a  pairwise  comparison 
matrix  for  each  alternative  on  every  criterion.  The  values  assigned  in  the  matrix  describe 
the  decision  maker’s  preference  between  two  alternatives  on  a  specific  criterion  (Ragsdale, 
1997).  Table  6  provides  a  preference  scale  used  for  pairwise  comparison  in  AHP.  This 
scale  is  similar  to  the  contracting  scale  for  numerical  and  adjectival  scoring  systems  shown 
in  Table  2  (paragraph  2. 2. 2. 2. 2). 
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Table  6 


Scale  for  Pairwise  Comparison  in  the  Analytical  Hierarchy  Process 


(Ragsdale,  1997:  718) 


Value 

Preference 

1 

Equally  Preferred 

2 

Equally  to  Moderately  Preferred 

3 

Moderately  Preferred 

4 

Moderately  to  Strongly  Preferred 

5 

Strongly  Preferred 

6 

Strongly  to  Very  Strongly  Preferred 

7 

Very  Strongly  Preferred 

8 

Very  Strongly  to  Extremely  Preferred 

9 

Extremely  Preferred 

The  AHP  process  begins  by  determining  the  relative  importance  of  the  criteria  in 
meeting  the  goal  and  their  relative  weights.  Pv  identifies  the  extent  alternative  i  is 
preferred  to  alternative  j.  Figure  5  provides  an  example  of  a  pairwise  comparison  matrix 
for  three  alternatives  X,  Y,  and  Z  in  relation  to  a  single  criterion,  price.  The  value  1  is 
placed  diagonally  along  the  matrix  to  indicate  that  an  alternative  evaluated  against  itself  is 
equally  preferred  because  it  is  the  same  alternative.  Using  the  scale  in  Table  6,  a  decision 
maker  would  select  his/her  preference  between  X  and  Y,  or  PX7 ;  between  X  and  Z,  or  Pxzi 
and  between  Y  and  Z,  or  P^.  In  the  scenario  in  Figure  5,  Pxy  =  5  because  the  decision 
maker  strongly  prefers  X  to  Y;  Pxz  =  7  because  the  decision  maker  very  strongly  prefers  X 
to  Z;  and  Pyz=  3  because  the  decision  maker  moderately  prefers  Y  to  Z.  To  determine  the 
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values  of  the  decision  maker’s  preferences  between  Y  and  X,  or  P^;  Z  and  X,  or  P^;  and 
Y  and  Z,  or  Pzy,  the  reciprocals  of  Pxy ,  Pxz,  and  Pyz  are  used.  The  reciprocal  formula  is  P,, 
=  1/  P?  (Ragsdale,  1997). 

The  next  step  in  AHP  is  to  normalize  the  pairwise  comparison  matrix.  First,  each 
column  in  the  matrix  is  summed,  then  each  entry  is  divided  by  its  column  sum  and  entered 
in  the  normalized  matrix  comparisons  table.  The  average  of  each  row,  i.e.  for  alternative 
X,  (0.745  +  0.789  +  0.636)/3  =  0.724,  indicates  the  relative  value  of  the  three  alternatives 
to  the  decision  maker  in  respect  to  the  price  criterion.  In  Figure  5,  X  is  the  most  preferred 
alternative  because  it  has  the  highest  price  score  (Ragsdale,  1997). 


X 

Y 

Z 

X 

1.000 

5.000 

7.000 

Y 

0.200 

1.000 

3.000 

Z 

0.143 

0.333 

1.000 

Sum 

1.343 

6.333 

11.00 

Normalized  Comparisons 

Price 

Consistency 

X 

Y 

z 

Scores 

Measure 

X 

0.745 

0.789 

mgm 

0.724 

3.141 

Y 

0.149 

0.158 

WSSm 

0.193 

3.043 

Z 

0.106 

0.053 

0.091 

3.014 

Consistency 

0.0567 

Ratio: 

Figure  5.  Price  Criterion  Pairwise  and  Normalized  Comparison  Matrices 
(Ragsdale,  1997:  721). 

A  consistency  measure  (Figure  5)  permits  the  decision  maker  to  verify  consistency 
in  preference  ratings  he/she  has  assigned  in  the  pairwise  comparison  matrix.  For  each 
alternative,  the  price  scores  obtained  from  the  normalized  matrix  are  multiplied  by  the 
preferences  assigned  in  the  rows  of  the  pairwise  comparison  matrix.  The  products  are 
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totaled  and  then  divided  by  the  price  score  for  the  particular  alternative.  For  example, 
alternative  X  would  calculate  (0.724  x  1)  +  (0.193  x  5)  +  (0.083  x  1)10.124  =  3.141; 
alternative  Y  would  calculate  (0.724  x  .2)  +  (0.193  x  1)  +  (0.083  x  3)/0. 193  =  3.043;  and 
alternative  Z  would  calculate  (0.724  x  0. 143)  +  (0. 193  x  0.333)  +  (0.083  x  l)/0.083 
=3.014  (Ragsdale,  1997:  720).  If  the  decision  maker  is  perfectly  consistent  in  assigning 
his/her  preferences,  each  consistency  measure  will  equal  the  number  of  alternatives  in  the 
problem.  In  the  problem  in  Figure  5,  n  =  3,  some  inconsistency  is  demonstrated  in  the 
stating  of  preferences,  but  some  inconsistency  is  acceptable  as  it  is  hard  for  a  decision 
maker  to  be  perfectly  consistent  when  assigning  preferences  between  a  large  number  of 
pairwise  comparisons.  The  scores  obtained  in  the  normalized  matrix  are  fairly  accurate  if 
the  inconsistencies  are  not  excessive  (Ragsdale,  1997:  719-721).  A  Consistency  Index 
(Cl)  and  Consistency  Ratio  (CR)  are  used  to  determine  whether  an  inconsistency  is 
excessive.  The  formulas  for  Cl  and  CR  are: 

Consistency  Index  (Cl)  =  (X  -  n)/n  -  1 

Consistency  Ratio  (CR)  =  CI/RI 

where: 

X  =  the  average  consistency  measure  for  all  alternatives 
n  =  the  number  of  alternatives 

RI  =  the  appropriate  random  index  values  [from  Table  7] 

(Ragsdale,  1997:  721) 

“If  a  pairwise  comparison  matrix  is  perfectly  consistent,  X  =  n  and  the  consistency 
ratio  =  0”  (Ragsdale,  1997:  722).  The  RI  values  in  Table  7  indicate  the  average  value  of 
Cl  if  all  the  preference  ratings  entered  in  the  pairwise  comparison  matrix  were  chosen 
randomly,  providing  that  all  diagonal  entries  equal  1  and  Py  =  1/  P,,.  If  CR  <  0. 10,  the 
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degree  of  inconsistency  is  considered  satisfactory;  if  CR  >0.10,  excessive  inconsistencies 
might  exist  and  render  the  AHP  results  meaningless  (Ragsdale,  1997:  722). 

Table  7 

Values  of  RI  for  AHP 
(Ragsdale,  1997:  722) 


u 

£1 

2 

0.00 

3 

0.58 

4 

0.90 

5 

1.12 

6 

1.24 

7 

1.32 

8 

1.41 

The  preceding  procedures  are  used  repeatedly  to  obtain  the  scores  for  each 
criterion.  Once  an  individual  score  is  obtained  for  all  criteria,  the  decision  maker 
determines  the  weights  to  indicate  the  relative  importance  of  each  criteria  in  relation  to  the 
other. 

The  pairwise  comparison  matrix  can  also  be  used  to  establish  and  compare  the 
criterion  weights.  Figure  6  establishes  weights  to  compare  three  criterion:  price,  support, 
and  ease  of  use  (Ragsdale,  1997:  724).  The  last  step  in  analyzing  an  AHP  decision 
problem  is  to  calculate  the  weighted  average  score  for  each  decision  alternative.  Figure  7 
shows  a  final  AHP  scoring  model  solution.  Based  upon  the  results  of  the  model, 
alternative  Y  has  the  highest  average  score  (Ragsdale,  1997:  725). 


Price 

Support  Ease  of  Use 

Price 

1.000 

0.333 

0.250 

Support 

3.000 

1.000 

0.500 

Ease  of  Use 

4.000 

2.000 

1.000 

Sum 

8.000 

3.333 

1.75 

Normalized  Comparisons 

Criterion 

Consistency 

Price 

Support  Ease  of  Use 

Weights 

Measure 

Price 

0.125 

0.143 

0.123 

3.006 

Support 

0.375 

0.286 

0.320 

3.018 

Ease  of  Use 

0.500 

0.571 

0.557 

3.030 

Consistency  Ratio: 

0.016 

Figure  6.  Criterion  Weights  Pairwise  and  Normalized  Comparison  Matrices 
(Ragsdale,  1997:  724). 


According  to  the  Vickery  (1989)  and  Noffsinger  (1991),  AHP  was  the 
recommended  choice  for  multi-criteria  decision-making  in  a  decision  support  system.  As 
previously  stated,  Vickery  and  Noffsinger  advocate  AHP  as  a  basis  for  designing  a 
decision-making  model  for  use  in  the  Government  source  selection  process  (Vickery,  1989 
and  Noffsinger,  1991). 


Criterion 

X 

Alternative 

Y 

Z 

Criterion 

Weights 

Price 

0.724 

0.193 

0.083 

0.123 

Support 

0.230 

0.648 

0.122 

0.320 

Ease  of  Use 

0.164 

0.297 

0.539 

0.557 

Weighted  Avg.  Score 

0.254 

0.397 

0.35 

1.000 

Figure  7.  Final  AHP  Scoring  Model  (Ragsdale,  1997:  725). 


2.4. 7  User  Interface.  A  user  interface  allows  the  user  to  control  which  data  and 
models  to  use  in  the  analysis.  The  user  interface,  the  connection  between  man  and 
computer,  should  be  designed  to  allow  the  user  to  input  and  extract  data  as  efficiently, 
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accurately,  and  quickly  as  possible.  The  user  interface  is  the  key  to  obtaining  the 
necessary  information  from  the  databases  within  the  system.  With  proper  planning,  a  user- 
friendly  interface  could  be  designed  to  interact  with  the  system  to  allow  the  user  to  sort, 
analyze,  and  manipulate  the  data  coming  into  and  flowing  through  the  system  without 
requiring  the  user  to  know  the  mechanics  of  the  system  (Scott,  1998).  The  Standard 
Procurement  System  (SPS)  mandated  for  use  by  DoD  acquisition  offices  will  become  the 
user  interface  to  electronically  access  the  various  information  systems  required  by  DoD 
acquisition  personnel.  Using  the  SPS,  the  user  will  be  able  to  make  inquiries  into  the  DSS 
and  ES  and  be  able  to  manipulate  and  analyze  data  flowing  into  and  out  of  the  model  base. 
2.5  Conclusion 

The  SSA  has  broad  discretion  in  selecting  the  evaluation  factors  that  apply  to  a 
specific  acquisition  and  their  relative  importance  to  the  total  mission  requirement,  except 
for  price  or  cost  which  is  always  a  consideration.  The  SSA  establishes  the  relative 
importance  of  each  criterion  through  the  use  of  decisional  rules  (weighting  the  criteria, 
prioritizing  or  trade-off  statements,  or  judgmental  decisional  rules).  To  evaluate  the 
criteria,  various  scoring  models  are  used  to  rate  an  offeror’s  proposal  including  color  or 
adjectival  ratings,  numerical  weights,  and  ordinal  rankings.  The  AF  only  restricts 
AFFARS  Appendix  AA  and  BB  source  selections  to  the  use  of  adjectival  “color  code” 
ratings.  For  any  competitive  acquisition  below  the  AFFARS  source  selection  threshold 
and  not  using  sealed  bidding  procedures,  there  is  no  prescribed  methodology  for  rating  as 
long  as  it  is  rational  and  applied  in  good  faith.  The  use  of  a  spreadsheet  program,  such  as 
Microsoft  Excel,  provides  numerous  tools  to  develop  a  variety  of  evaluation  models 
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including  LP  optimization,  multi-criteria  scoring,  and  AHP.  These  models  can  provide  a 
structured  approach  to  the  decision-making  process. 

In  addition,  information  technology  systems,  such  as  DSSs  and  ESs,  are 
continually  improving  with  advancing  technology.  Research  has  demonstrated  these 
systems  are  being  designed  using  multiple  configurations  to  accommodate  various  types  of 
business  applications.  With  the  right  configuration,  this  advanced  technology  has  the 
potential  to  assist  the  SSA  in  obtaining  and  analyzing  more  complete  information  through 
the  collection  of  data  into  shared  data  warehouses.  The  use  of  DSSs  and  ESs  in  the 
source  selection  process  can  permit  the  SSA  to  obtain  expert  advice  and  decision  analysis 
support  on  an  ad  hoc  basis.  The  inclusion  of  management  science  tools,  such  as  Microsoft 
Excel,  within  the  model  base  of  an  information  system  can  transform  the  system  from  a 
tactical  system  into  a  strategic  system. 

The  following  chapter  discusses  the  methodology  behind  the  development  of  the 
theoretical  schematic  model  of  using  information  technology  to  manipulate  and  analyze 
information  flowing  into  the  source  selection  process  to  assist  with  decision-making.  The 
methodology  behind  the  design  and  development  of  the  Microsoft  Excel  baseline  models  is 
also  discussed  in  Chapter  3.  The  LP  optimization,  multi-criteria  scoring,  and  AHP  tools 
are  the  focus  of  the  design  and  development  of  the  baseline  models. 
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3.  Methodology 


3.1  Introduction 

This  chapter  outlines  the  data  collection  and  analysis  method  used  to  answer  the 
investigative  questions  previously  posed.  Answering  these  questions  allows  assessment 
of  the  strategic  role  of  information  technology  to  assist  the  SSA  evaluate  and  analyze 
multiple  alternatives  during  the  decision-making  process  of  a  competitive  source  selection. 

3.1.1  Investigative  Questions.  The  investigative  questions  are: 

1)  Can  a  schematic  model  be  designed  to  identify  areas  where  information 
technology  can  improve  the  flow  of  information  into  the  source  selection  evaluation 
process  and  expand  the  present  tactical  approach  to  evaluation  of  alternatives  into  a  more 
strategic  approach? 

2)  Can  a  baseline  model  be  designed,  based  upon  Microsoft  Excel  SOLVER,  to 
assist  the  SSA  in  choosing  the  best  alternative  source  in  a  competitive  source  selection? 

3.1.2  General  Approach.  The  primary  methods  of  solving  the  research  problem 
include  a  combination  of  case  study  and  grounded  theory  approaches.  The  case  study 
approach  was  used  in  three  ways.  It  was  used  in  collecting,  organizing,  and  reporting  the 
information  on  the  boundaries  set  by  contracting  source  selection  policies  and  regulations. 
It  was  used  to  examine  the  types  of  information  technology  available  to  automate  the  flow 
of  information  into  the  source  selection  process  while  remaining  within  contracting 
boundaries.  Finally,  case  studies  were  used  to  analyze  literature  relating  to  the  various 
modeling  techniques  contained  in  Microsoft  Excel  to  identify  which  techniques  are  best  for 


72 


use  in  the  source  selection  process.  Additionally,  the  case  study  approach  was  used  as  the 
basis  for  the  development  of  the  schematic  information  model. 

A  grounded  theory  approach  was  used  as  the  basis  for  development  of  the 
Microsoft  Excel  SOLVER  models.  Data  from  historical  contract  files  was  systematically 
gathered  and  analyzed  as  the  models  were  being  developed.  As  the  continuous  inflow  of 
data  affected  each  model’s  development,  its  design  was  re-analyzed  and  modified 
accordingly. 

3.2  Schematic  Model  Development 

The  schematic  information  flow  model  was  developed  after  an  extensive  literature 
review  of  primary  and  secondary  data  sources  on  the  topics  of  the  acquisition  process, 
information  technology  systems,  and  management  science  techniques.  Commercial 
information  systems  used  in  decision-making  and  in  evaluating  and  analyzing  multiple 
criteria  decisions  were  reviewed  to  determine  the  most  feasible  information  technology 
solutions  to  automate  the  flow  of  information  required  by  the  SSA  to  make  a  sound 
decision.  Information  flowing  into  the  source  selection  process  was  analyzed  for  its 
applicability  to  assist  the  SSA  in  making  an  optimal  decision,  and  its  potential  for 
automation.  A  theoretical  model  based  upon  system  application,  benefits,  features,  and 
capabilities  was  designed  to  flow  information  into  the  Microsoft  Excel  SOLVER  model 
base  for  analysis. 

3.3  Microsoft  Excel  SOLVER  Decision  Model 

Design  and  development  of  a  baseline  SOLVER  spreadsheet,  was  addressed  by 
focusing  on  the  general  evaluation  criteria,  decisional  rules,  and  scoring  models  used  as 
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the  basis  for  awards  in  sealed  bidding,  competitive  proposals,  and  AFFARS  Appendix 
AA/BB  source  selections.  Current  contract  policy  and  regulations  on  the  evaluation  and 
award  process  (paragraphs  2. 1  -  2.2)  defined  the  boundaries  of  the  basic  models 
developed.  At  the  Aeronautical  Systems  Center,  Wright-Patterson  AFB  OH,  permission 
was  granted  by  the  SSA  for  DAC  Programs  and  Other  Contracting  acquisitions  to  search 
archived  contract  files  for  data  pertinent  to  the  development  of  a  baseline  model. 

Authority  was  not  granted  to  access  contract  files  using  AFFARS  Appendix  AA  formal 
source  selection  procedures  so  no  Appendix  AA  contract  data  was  used  to  develop  the 
baseline  models.  Contract  files  were  identified  and  located  using  the  current  contracting 
Automated  Management  Information  System  (AMIS).  Contracts  were  “tagged”  for  use 
by  special  identifiers  within  the  AMIS  system.  These  identifiers  indicated  whether  the 
contract  was  awarded  as  a  competitive  or  sole  source  acquisition;  and  whether  the  use  of 
sealed  bid,  competitive  proposal,  or  formal  source  selection  procedures  were  used. 
Contract  files  using  AFFARS  Appendix  BB  procedures  were  reviewed  at  ASC/SYTffi^ 
Source  Selection  facility.  At  ASC/PKW,  the  managers  of  the  construction  and  specialized 
sendees  branches  manually  identified  and  provided  appropriate  contract  files  for  review. 
These  two  branches  were  selected  due  to  the  nature  of  the  acquisitions.  Construction 
acquisitions  normally  used  sealed  bidding  procedures,  and  specialized  services  normally 
use  FAR  Part  15  procedures  implementing  the  lowest  priced  technically  acceptable  basis 
of  award.  At  ASC/PKW,  contract  files  using  sealed  bidding,  FAR  Part  15  competitive 
procedures,  and  some  AFFARS  Appendix  BB  source  selections  were  reviewed. 
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Data  collected  from  the  contract  files  answered  the  following  questions: 

1 .  What  type  of  acquisition  procedures  were  used?  Sealed  Bid,  Competitive 
Procurement,  AFFARS  Appendix  BB. 

2.  What  was  the  basis  for  award?  Lowest  priced  bid.  Lowest  priced,  technically 
acceptable.  Best  value. 

3 .  What  was  the  structure  and  number  of  Contract  Line  Items  (CLIN)  in  the 
solicitation?  And  what  was  the  type  of  contract,  i.e.  basic  IFB,  RFP,  or 
requirements/indefinite  delivery,  indefinite  quantity  contract? 

4.  What  were  the  evaluation  criteria  in  the  solicitation? 

5.  What  type  of  constraints  were  mandated  by  the  solicitation,  i.e.  single 
aggregate  or  multiple  awards,  small  business  set-aside,  responsive  or 
responsible  contractor,  FOB  destination/origin? 

6.  How  many  contractors  submitted  an  offeror? 

7.  What  did  each  contractor  propose? 

8.  What  were  the  results  of  the  evaluations,  i.e.,  color  codes,  risk  ratings? 

9.  What  was  the  SSA  decision? 

General  trends  were  identified  for  each  type  of  acquisition  procedure,  and  this  type 
of  data  was  incorporated  into  the  baseline  models  during  spreadsheet  design  to  add 
realism  to  the  models.  Common  constraints  were  identified  for  each  type  of  product 
acquisition  to  assist  in  determining  ease  of  modifiability  between  the  models.  Once  the 
baseline  models  were  complete,  usability  and  accuracy  of  each  spreadsheet  model  was 
verified  and  validated  by  incorporating  actual  data  collected  from  competitive  bids  and 
proposals.  The  verification/validation  phase  also  helped  identify  benefits/advantages  and 
potential  problems/disadvantages  of  these  models. 
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3.3.1  Baseline  Models.  The  baseline  models  were  designed  using  mathematical 
programming  techniques  and  the  Microsoft  Excel  SOLVER  software  program.  Integer 
Linear  Programming  (ILP),  and  Multi-criteria  Scoring/AHP  methods  were  identified  as 
the  most  promising  methods  for  model  development.  Four  steps  were  used  to  develop  the 
ILP  optimization  models: 

1 .  Organize  the  data  for  the  model  on  the  spreadsheet. 

2.  Reserve  separate  cells  in  the  spreadsheet  to  represent  each  decision  variable  in 
the  model. 

3 .  Create  a  formula  in  a  cell  in  the  spreadsheet  that  corresponds  to  the  objective 
function  in  the  model. 

4.  For  each  constraint,  create  a  formula  in  a  separate  cell  in  the  spreadsheet  that 
corresponds  to  the  left-hand  side  (LHS)  [and  right-hand  side  (RHS)]  of  the 
constraint.  (Ragsdale,  1997:  45). 

A  single  baseline  using  ILP  was  developed  for  sealed  bidding  acquisitions  since  the 
technical  nature  of  sealed  bid  evaluations  was  less  complex.  A  separate  spreadsheet  using 
AHP  was  used  to  design  the  baseline  for  formal  source  selections.  Due  to  the  variety  of 
evaluation  criteria  options  available  for  use  in  FAR  Part  1 5  competitive  acquisitions,  the 
two  baselines  (ILP  and  AHP)  were  then  modified  into  a  single  model  to  determine  if  they 
could  meet  the  requirements  for  competitive  acquisitions  procedures. 

3.3.2  Baseline  for  Sealed  Bidding.  An  understanding  of  the  “lowest  bid”  basis 
for  award  in  sealed  bidding  acquisitions  (FAR  Part  14)  was  required  for  development  of 
this  baseline.  The  evaluation  criteria  and  data  on  sealed  bidding  described  in  paragraph 
2.2. 1  was  used,  in  conjunction  with  actual  data  from  contract  data  files,  to  determine  the 
best  approach  for  model  design.  ILP  was  used  to  develop  the  sealed  biding  model. 

3. 3. 3  Baseline  for  AFFARS  Appendix  AA/BB  Source  Selections.  An 
understanding  of  best  value  awards  was  required  for  the  design  and  development  of  the 
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baseline  for  AFFARS  Appendix  AA/BB  formal  source  selections.  Both  AFFARS 
Appendix  AA  and  BB  source  selections  use  the  evaluation  procedures  identified  in 
paragraph  2.2.2.3  so  a  single  model  was  developed  for  use  by  both  types  of  source 
selections.  Using  the  Microsoft  Excel  software,  model  data  was  examined  and  input  using 
the  multi-scoring  and  analytical  hierarchy  approaches  for  multi-criteria  decision-making. 
The  AF  requires  technical  data  to  be  evaluated  using  color  ratings  (blue,  green,  yellow, 
and  red).  Quantitative  measures  were  assigned  to  colors  to  measure  the  differences  in  the 
descriptive  data.  Since  past  performance  data  is  also  evaluated  using  color  ratings,  the 
same  measurement  scheme  was  used.  Pairwise  preference  matrices  were  developed  at  the 
lowest  possible  level  to  permit  consolidation  of  input  from  multiple  technical  evaluators. 
The  model  was  designed  to  permit  the  flow  of  information  to  higher  levels  of  evaluation 
(factor,  subfactor).  Once  the  technical  data  and  past  performance  data  were  incorporated 
into  the  decision  model  using  the  color  coded  rating  scale,  the  cost  data  was  incorporated 
using  a  preference  scale  similar  to  Table  6.  A  pairwise  matrix  for  the  criterion  weights 
was  developed,  in  to  incorporate  the  technical,  past  performance,  and  cost  data,  into  a 
final  AHP  pairwise  matrix  for  determination  of  the  best  value  alternative.  The  evaluation 
criteria  and  data  on  AFFARS  Appendix  AA/BB  described  in  paragraph  2.2. 1  were  used, 
in  conjunction  with  actual  data  from  contract  data  files,  to  determine  the  best  approach  for 
model  design.  Due  to  a  wide  variation  among  contracts  on  the  basis  of  award  and/or 
selected  evaluation  criteria,  the  model  was  designed  generically  for  flexibility  and  easy 
adjustments. 
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3.3.4  Baseline  for  Competitive  Proposals.  The  baseline  for  the  Competitive 
Proposals  (FAR  Part  15),  was  developed  last  as  the  evaluation  procedures  could  be  similar 
to  either  those  used  to  develop  the  AFFARS  Appendix  AA/BB  formal  source  selection 
baseline  or  the  sealed  bidding  baseline.  The  main  differences  to  consider  were  basis  of 
award,  and  the  rating  methods  applied  to  evaluation  of  the  technical  criteria.  The  color 
rating  system  was  not  mandatory  for  competitive  proposal  acquisitions  which  did  not  meet 
the  thresholds  of  AFFARS  Appendix  AA/BB  source  selections  so  the  other  rating 
methods  described  in  paragraph  2.2  were  optional  for  use  by  the  SSA.  A  small  number  of 
contract  files  were  reviewed  to  determine  which  rating  methods  were  commonly  used  to 
evaluate  the  technical  and  past  performance  data,  and  how  to  incorporate  this  information 
into  the  model.  The  lowest  priced  technically  acceptable  basis  of  award  appeared  to  be 
most  frequently  used  in  FAR  Part  1 5  acquisitions.  A  common  rating  method  for  these 
acquisitions  was  to  rate  the  technical  proposals  on  the  basis  of  Acceptable,  Marginal,  or 
Unacceptable.  Due  to  this,  the  ILP  optimization  model  was  selected  for  review.  The 
technical  data  were  incorporated  using  binary  and  linking  constraints  into  the  decision 
model,  and  then  merged  with  the  cost  data  to  determine  the  optimal  award  based  upon  the 
award  lowest  priced  technically  acceptable  criteria 

3.3.5  Model  Verification.  To  achieve  a  logical  spreadsheet  design,  the  following 
goals  were  considered  during  the  creation  of  each  baseline  model,  and  were  used  as  the 
basis  to  evaluate  the  final  models  developed: 

1 .  Communication  -  The  primary  design  objective  of  a  model  is  to  communicate 
relevant  aspects  of  the  problem  in  a  clear  and  intuitively  appealing  manner. 

2.  Reliability  -  The  output  of  the  spreadsheet  should  be  correct  and  consistent. 
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3.  Auditability  -  The  model  should  be  setup  in  an  intuitively  appealing,  logical 
layout  to  allow  readers  to  retrace  the  steps  followed  to  generate  different 
output.  This  will  assist  with  ease  of  understanding  and  verification. 


4.  Modifiability  -  The  data  and  assumptions  in  which  the  spreadsheets  are  built 
will  change  frequently.  To  accommodate  for  changing  user  requirements,  the 
spreadsheet  should  be  designed  for  easy  modifications  or  enhancements. 
(Ragsdale,  1997:  59) 

Communication  was  examine  regarding  three  points:  how  effective  the  model’s 
design  was  for  presenting  final  results  to  the  SSA,  how  effective  was  the  spreadsheet’s 
formatting  to  communicate  the  mechanics  of  the  model  to  users  using  colors  and  other 
techniques.  Reliability  was  examined  for  accuracy,  consistency,  and  effectiveness  of 
spreadsheet  design,  AHP  and  ILP  techniques,  and  the  Microsoft  Excel  SOLVER  program. 
Auditability  was  examined  for  ability  to  retrace  data  output  to  its  source  of  input  between 
worksheets,  ability  to  track  and  update  formulas,  and  formatting  techniques  used  to 
identify  common  features  among  input.  Modifiability  was  examined  for  ability  to  generate 
new  models  using  copy/paste/edit  techniques,  ability  to  modify  the  spreadsheet  to 
accommodate  new  or  changed  acquisition  data,  such  as  increases  or  decreases  in  the 
number  of  offerors,  CLINs,  or  evaluation  criteria;  changes  resulting  from  clarifications  and 
discussions  with  offerors,  and  changes  in  the  number  of  limitations  or  constraints. 

3.4  Conclusion 

The  techniques  identified  in  this  chapter  were  used  to  collect,  organize,  and  analyze 
the  data  used  to  develop  the  schematic  and  baseline  models  constructed  in  this  study.  The 
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following  chapter  describes  the  design  and  development  of  the  models  and  the  resulting 
analysis. 
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4.  Model  Development  and  Analysis 


This  Chapter  presents  the  findings  and  analysis  of  the  models  developed  during 
this  research.  The  schematic  model  is  a  theoretical  model  of  how  the  source  selection 
process  could  be  automated  to  collect,  manipulate,  and  analyze  information  to  assist  the 
source  selection  decision-making  process.  The  models  developed  were  designed  within 
the  constraints  placed  upon  the  current  source  selection  process  by  contracting 
regulations,  laws,  and  policies.  A  more  effective  and  efficient  model  may  be  feasible  if 
waivers  were  granted  to  deviate  from  current  regulations  regarding  adjectival  color 
coding,  or  the  allowability  to  compare  an  offeror’s  proposal  against  its  competitors. 

4.1  Investigative  Questions 

1)  Can  a  schematic  model  be  designed  to  identify  areas  where  information 
technology  can  improve  the  flow  of  information  into  the  source  selection  evaluation 
process  and  expand  the  present  tactical  approach  to  evaluation  of  alternatives  into  a  more 
strategic  approach? 

2)  Can  a  baseline  model  be  designed,  based  upon  Microsoft  Excel  SOLVER,  to 
assist  the  SSA  in  choosing  the  optimal  or  best  alternative  source  in  a  competitive  source 
selection? 

4.2  Investigative  Question  Number  One:  Development  and  Analysis  of  a  Schematic 
Model 

At  the  conclusion  of  the  literary  research  on  information  technology  and  the  flow 
of  information  into  the  source  selection  process,  a  theoretical  model  (Figure  8)  of  a 
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Decision  Support  System,  using  an  “intranet”  philosophy,  was  designed  to  assist  the  SSA 
choose  the  best  alternative  for  contract  award.  An  intranet  is  a  computerized  network 
connecting  people,  data,  processes,  and  other  workings  of  a  particular  business  entity — 
DoD  procurement  offices — into  a  business- wide  web  (Lewis,  1998:  1).  The  complexity  of 
the  Government  source  selection  process  requires  SSAs  to  be  able  to  analyze  and 
manipulate  data  in  an  ambiguous  and  complex  environment.  This  theoretical  model  could 
be  used  to  evaluate  sealed  bids,  competitive  proposals,  or  AFFARS  Appendix  AA/BB 
source  selections,  however  the  following  discussion  in  reference  to  Figure  8  is  limited  in 
use  to  AFFARS  Appendix  AA/BB  source  selections. 

By  the  year  2000,  offeror  cost  and  technical  proposals  will  be  delivered  to  the 
source  selection  team  through  the  use  of  an  EC/EDI  or  internet  system  (SAF/AQXA, 

1997).  The  DoD  SPS  acts  as  the  user  interface,  or  link,  between  the  SSAs  and  the 
information  stored  within  the  various  shared  data  warehouses.  Its  use  of  relational 
databases  links  the  data  within  the  system  and  should  allow  the  source  selection  team  to 
conduct  ad  hoc  queries  during  the  decision-making  process.  The  source  selection  team, 
through  operation  of  the  user  interface,  can  access  the  inference  engine  to  obtain  advice, 
regulation,  and  historical  information.  The  inference  engine  can  be  used  to  solve  problems 
before  and  during  a  source  selection  by  accessing  and  analyzing  information  from  the  DSS 
knowledge  base.  The  knowledge  base  in  the  theoretical  model  consists  of  two  expert 
systems  and  an  additional  DSS  within  the  knowledge  base  to  form  an  “expert  shell.”  The 
additional  DSS  contains  historical  records  of  source  selections,  lessons  learned 
information  obtained  during  previous  source  selections,  historical  data  on  contract  awards. 
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Figure  8.  Schematic  Model  of  Information  Technology,  Decision  Support  System, 
in  the  Source  Selection  Process. 


cost  and  pricing  evaluation  data  on  past  awards.  Government  cost  estimates  for  current 
source  selections,  funding  data  for  the  acquisition,  and  the  source  selection  evaluation 
criteria  and  standards  for  the  current  acquisition.  The  DSS  would  be  able  to  use  case- 
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based  reasoning,  together  with  “what  if’  capabilities,  to  allow  the  source  selection  team  to 
ask  ad  hoc  questions  which  are  unique  to  each  individual  acquisition.  In  conjunction,  an 
expert  system  with  AI  capabilities  is  linked  to  the  additional  DSS  to  act  as  a  consultant  by 
providing  expert  contracting,  source  selection,  and  legal  advice.  In  addition  to  the  rules 
and  opinions  from  experts  in  the  contracting  and  legal  field,  this  expert  system  contains  all 
of  the  contracting  rules,  regulations,  and  policies;  source  selection  regulations;  and 
acquisition  iegal  cases,  such  as  General  Accounting  Office  (GAO)  case  reports  and  protest 
court  cases.  A  second  ES  with  AI  capabilities,  is  used  to  provide  technical  advice  for 
program  personnel  and  engineers  during  technical  evaluations.  Rules  and  expert  opinion 
on  factors  such  as  engineering  design  and  development  techniques,  production  and 
process  control,  reliability  and  maintainability  measures,  technical  data  and  drawings  of 
existing  systems,  logistics  and  transportation  data,  and  standards  for  the  number  of  hours 
to  perform  a  task  are  contained  in  the  database  of  this  expert  system. 

The  AI  expert  systems  within  the  knowledge  base  would  allow  the  inference 
engine  to  deduce  information  and  also  ask  questions  regarding  information  that  it  couldn’t 
analyze.  The  source  selection  team  could  use  the  DSS  at  this  point  to  ensure  each 
proposal  has  complied  with  contracting  and  legal  policies.  The  knowledge  base  can  be 
used  by  the  inference  engine  to  assist  the  evaluation  team  compare  each  offerors  proposal 
against  the  minimum  mandatory  requirements  (paragraph  2. 2. 2. 3. 1)  to  determine  minimum 
compliance  with  requirements.  If  minimum  mandatory  requirements  are  used  to  determine 
the  competitive  range,  the  DSS  could  provide  an  initial  evaluation  of  proposals  and  rate 
them  on  a  pass/fail  or  green/red  basis  by  determining  which  offerors  met  the  mandatory 


requirements.  The  source  selection  team  could  then  use  the  data  provided  by  the  DSS  to 
determine  which  proposals  continue  into  detailed  evaluations.  Proper  planning  during  the 
pre-solicitation  phase  of  the  acquisition  would  be  required  to  solidify  the  minimum 
standards  required  to  meet  the  evaluation  requirements  established  to  allow  the  computer 
to  scan  each  proposal  to  determine  compliance.  Once  the  SSA  agrees  with  the  analysis  of 
the  competitive  range  determination,  the  system  could  generate  notices  to  the  unsuccessful 
offerors  (proposals  rated  “red”)  and  notify  them  electronically  via  EC/EDI  or  internet. 

If  minimum  mandatory  requirements  were  not  pre-established,  the  source  selection 
team  could  begin  by  evaluating  the  Essential  Characteristics  (paragraph  2.2.2. 3.1)  of  the 
proposals  using  the  Microsoft  Excel  Model  Base.  When  evaluating  the  Essential 
Characteristics,  the  AF  requires  the  use  of  the  adjectival  color  coding  scheme  (blue,  green, 
yellow,  red)  to  determine  how  well  each  offeror  rates  in  comparison  to  the  pre-established 
evaluation  criteria.  The  team  assesses  criteria  at  the  element  level  using  symbols  (+,V,  -), 
and  proposal  and  performance  risk  using  descriptive  adjectives  (high,  moderate,  low). 

The  DSS  could  be  used  for  advisory  assistance  during  this  portion  of  the  technical 
evaluation. 

Proposal  cost  and  pricing  data  could  be  analyzed  for  realism  and  reasonableness 
using  the  historical  data  and  the  Government  estimate  in  the  DSS.  The  technical  team 
could  use  the  technical  expert  system  to  obtain  information  on  labor  standards  to  calculate 
the  number  of  hours  and  the  skill  mix  required  to  successfully  complete  a  task.  This 
would  provide  a  more  objective  basis  for  the  evaluations,  and  also  assist  the  technical  team 
assess  the  soundness  and  realism  of  the  offerors  technical  approach.  The  historical  files 
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could  be  searched  for  labor  rates,  material  costs,  and  other  relevant  costs  to  assist  in 
determining  the  reasonableness  of  the  offer.  A  link  to  the  internet  could  also  be 
incorporated  to  obtain  current  pricing  on  materials,  as  many  companies  are  now 
advertising  numerous  products  and  services  on  General  Service  Agency  (GSA) 
government  pricing  schedules  on  the  World  Wide  Web.  Expert  audit  advice,  along  with 
DCAA  audit  reports,  could  be  included  in  the  DSS  database  to  permit  queries  regarding 
overhead,  general  and  administration,  profit,  effects  of  proposed  changes  in  contract  type 
on  costs,  and  risk  analysis.  Statistical  graphics  could  assist  examination  of  front-end 
loaded  or  unbalanced  bids  by  showing  each  offeror’s  costs  in  relation  to  the  Government 
estimate  and  its  competitors. 

Past  performance  data,  analyzed  and  obtained  from  the  shared  data  warehouse, 
could  be  used  in  conjunction  with  sources  of  past  performance  information  provided  by 
the  offeror,  to  determine  risk  associated  with  offeror  performance  based  upon  an 
integrated  assessment  of  the  offeror’s  past  history.  Until  recently,  past  performance  was 
normally  considered  a  general  consideration  during  the  evaluation  process,  but  has  now 
increased  in  importance.  It  is  now  usually  considered  a  specific  evaluation  criteria. 

To  conduct  statistical  analysis  on  the  data  within  the  past  performance  data 
warehouse,  an  on-line  analytical  processing  component  would  be  required.  Input  of 
CPARs  data  in  the  correct  format  could  allow  the  system  to  analyze  the  data  to  show  past 
trends  in  contract  cost  over-runs  or  completion  under  cost,  on-time  or  late  deliveries  and 
schedule  compliance,  and  to  rate  the  quality  of  products  or  services  delivered,  i.e., 
compliance  with  specifications,  deviations  due  to  contractor  fault,  and  superior  or  above 
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minimum  quality.  Statistical  analysis  could  be  used  to  analyze  past  trends  and  to  predict 
and  assign  a  probability  rating  factor  on  future  performance.  The  data  warehouse  could 
provide  both  a  detailed  and  summary  report  on  the  analysis,  along  with  the  probability 
factor,  to  allow  the  source  selection  team  to  incorporate  additional  external  past 
performance  information  provided  by  the  contractor  or  other  sources  not  contained  within 
the  CPARS  system. 

Once  the  technical  and  past  performance  data  are  evaluated,  the  source  selection 
team  would  then  direct  this  information  into  the  model  base,  along  with  the  cost/price 
evaluation  information,  for  analysis  of  the  optimal  solution  to  determine  which  alternative 
offeror  is  best.  The  model  base  includes  various  scoring  models,  statistical  software, 
optimization  models,  and  software  necessary  for  the  analysis  of  the  proposed  alternatives. 
The  SPS  interface  would  need  to  permit  the  source  selection  team  to  input  and  analyze  the 
data  using  the  model  base  without  having  specific  computer  or  mathematical  expertise. 

The  model  base  could  construct  the  specific  evaluation  model  in  response  to  data  input  by 
the  source  selection  team,  i.e.,  computer  asks  number  of  offerors  to  be  evaluated,  and  the 
number  of  evaluation  criteria  and  preferences  (color  codes),  and  the  computer  builds  an 
evaluation  matrix  based  upon  user  response.  The  model  base  incorporated  in  this  DSS 
model  contains  the  Microsoft  Excel  SOLVER  program.  Other  analytical  software 
programs  can  be  included  within  the  model  base  as  necessary.  Once  the  model  base  has 
evaluated  and  determined  the  optimal  solution,  the  information  is  sent  back  to  the  user  for 
source  selection  consideration.  Once  the  SSA  agrees  with  the  decision  provided  by  the 
team,  award  can  be  made  through  the  EC/EDI  or  internet. 
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A  simple  theoretical  model  for  less  complex  acquisitions,  such  as  sealed  bidding, 
might  consist  of  an  EC/EDI  or  internet  system  to  input  the  proposal  data  into  a 
Government  database,  which  in  turn  could  flow  the  relevant  information  into  the  model 
base,  or  optimization  program.  Since  the  SPS  system  uses  standardized  ANSI  XI 2 
documents,  the  relational  database  could  sort  data  from  specific  fields  in  the  proposal 
document  and  redistribute  it  into  the  model  base  using  Microsoft’s  Access  database 
features.  The  system  could  then  analyze  the  data  for  inconsistencies  (i.e.,  error  in  the 
addition  of  proposed  unit  prices  and  quantities  to  the  extended  price)  and  notify  the 
contracting  official  for  potential  errors  or  abnormalities  in  the  proposal  data.  The 
contracting  official  could  visually  analyze  the  data  to  determine  if  the  offeror  would  be 
considered  responsive  or  eliminated  from  further  competition.  Once  the  contracting 
official  has  determined  the  data  is  correct,  the  proposal  information  could  be  electronically 
re-submitted  to  the  SOLVER  program  for  determination  of  the  optimal  solution. 

As  stated  earlier,  these  are  theoretical  models  developed  from  literary  research. 
Prior  to  development  and  implementation  of  a  model  for  contracting,  a  field  study  should 
examine  the  mechanics  of  actual  DSS  and  expert  systems  to  ensure  benefits  and  features 
cited  in  the  literature  provide  the  necessary  level  of  technical  operability  required  to 
support  a  source  selection.  This  field  research  is  important  as  a  system  designed  to  meet 
the  constraints  of  Government  contracting  regulations  and  laws  would  differ  from  the 
multi-criteria  evaluation  systems  designed  for  commercial  business  applications,  such  as 
selecting  which  product  to  take  into  full-scale  production.  The  technology  exists  to  meet 
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the  requirements  but  modifications  may  be  required  to  fully  implement  the  applications 
within  the  Government  contracting  field. 

4.3  Investigative  Question  Number  Two:  Development  and  Analysis  of  the  Microsoft 
Excel  SOL  VER  Baseline  Models 

The  Microsoft  Excel  Spreadsheet  Models  designed  and  developed  in  this  section 
represent  part  of  the  model  base  as  described  in  the  previous  section.  The  models  are 
developed  for  manual  input  by  a  contracting  official  as  no  interface  device  currently  exists 
to  provide  connectivity  with  receipt  of  electronic  data.  These  models  are  designed  to 
assist  the  decision-maker  evaluate  source  selection  information,  not  to  replace  the 
decision-maker. 

4.3.1  FAR  Part  14,  Sealed  Bidding  Procedures.  The  Microsoft  Excel  SOLVER 
program  using  ILP  was  used  to  design  the  model  for  the  Sealed  Bidding  Procedures,  FAR 
Part  14.  Windows  97  was  used  in  the  development  of  this  model.  Appendix  C,  Figures 
20  -  24  provide  visual  data  of  charts  from  actual  models  developed.  Appendix  C,  Table  8 
provides  the  SOLVER  parameters  and  options  necessary  to  solve  the  ILP  problem.  The 
sealed  bid  model  was  designed  based  upon  the  lowest  priced  responsive  bid  concept.  The 
model  was  arranged  to  permit  the  contracting  official  to  input  specific  background  data, 
and  to  receive  some  direction  from  the  initial  “Acquisition  Background”  worksheet.  Two 
SOLVER  models  were  designed  to  permit  award  of  either  a  single,  or  “aggregate” 
contract;  or  to  permit  split,  or  “multiple”  awards.  If  the  solicitation  permitted  multiple 
awards,  the  contracting  official  must  use  both  of  the  models  in  the  “Sealed  Bids  - 
Aggregate”  and  “Sealed  Bids  -  Multiple”  worksheets  to  allow  comparison  of  the  best 
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alternative  to  the  Government.  The  models  were  designed  to  reduce  the  amount  of  data 
entered  into  the  spreadsheet  by  the  contracting  official  by  using  ‘ -CELL  ADDRESS” 
formulas  in  the  price  and  quantity  cells  (Figure  20)  for  the  “multiple”  award  model.  This 
permitted  the  automatic  transfer  of  data  from  the  “Sealed  Bids  —  Aggregate  worksheet 
into  the  “Sealed  Bids  -  Multiple”  worksheet.  A  third  worksheet  “Unbalanced  Bids”  using 
graphics  was  provided  to  permit  analysis  of  offeror’s  proposals  for  indications  of 
unbalanced  bidding  or  front-end  loaded  bidding.  Fill-in  data  from  the  “Unbalanced  Bids” 
worksheet  would  be  transferred  to  both  the  “Sealed  Bids  -  Aggregate”  and  “Sealed  Bids  - 
Multiple”  worksheets. 

In  the  “Sealed  Bids  -  Aggregate”  worksheet,  the  contracting  official  must  input 
the  offerors  proposed  price  and  quantity  required  by  the  government.  After  review  of 
ASC/PKWO’s  construction  contracts  using  sealed  bidding  procedures,  it  was  determined 
that  a  model  incorporating  six  offerors  and  four  CLIN  would  be  adequate  for 
development.  Theoretical  prices  and  quantities  were  incorporated  into  the  model,  using 
numbers  easy  to  calculate  without  the  use  of  the  SOLVER,  to  permit  evaluation  of  the 
model’s  ability  to  provide  a  solution.  Formulas,  using  ‘ -SUMPRODUCT”  or  ‘ -SUM” 
were  entered  into  the  model  to  calculate  the  totals  of  the  columns  and  rows,  respectively, 
for  the  price  and  quantity  data.  The  target  cell,  or  the  cell  the  SOLVER  program  was  to 
minimize,  was  the  “Total  Price”  cell  (Figure  22).  An  “=SUM”  formula  was  entered  into 
the  target  cell.  Binary  constraints,  or  changing  cells,  and  Linking  constraints  (Figure  21) 
were  added  to  the  aggregate  award  model  to  restrict  the  SOLVER  from  selecting  more 
than  one  offeror  for  award.  This  restriction,  nor  the  corresponding  section  in  Figure  21, 
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was  not  required  for  the  SOLVER  to  determine  the  lowest  price  in  the  multiple  awards 
model,  as  it  searches  for  the  lowest  priced  CLIN  among  all  offerors  and  awards  multiple 
contracts  based  upon  lowest  overall  price.  Binary  constraints  were  used  to  incorporate 
information  regarding  solicitation  constraints  and  to  determine  whether  or  not  the  offerors 
complied  with  these  constraints  (Figure  23).  These  constraints  included  data  on  small 
business  restrictions,  unbalanced  bidding,  front-end  loaded  bidding,  and  offeror 
responsiveness;  however  the  method  devised  to  incorporate  these  constraints  could  be 
used  to  incorporate  numerous  other  constraints  mandated  within  the  acquisition  field. 

Each  qualitative  constraint  was  established  to  answer  a  specific  “yes”  or  “no”  question.  A 
“yes”  answer  was  then  converted  to  a  “1”  and  a  “no”  answer  was  converted  to  a  “0.”  A 
binary  summary  answering  the  question  “Was  the  requirement  met?”  was  created  and 
were  linked  to  the  SOLVER  program  using  the  formula  “=IF(AND(BSi,  BS2,  BSs, 
BS4,BS5),  1,0),  where  BS  =  binary  summary.”  This  formula  translated  means  “if  all  of  the 
five  constraints  are  true  (reflected  by  a  “1”  in  the  model),  then  input  “1,”  otherwise  “0.” 
Using  the  constraint  D33:I33  <  D  109:1 109,  or  the  “Binary:  Contractor  Used”  cells  (Figure 
21)  must  be  less  than  or  equal  to  the  “If  all  constraints  are  met,  then  =  1,  if  not  met  then 
0”  (Figure  25).  This  constraint  prevented  an  offeror  who  did  not  comply  with  the 
requirements  of  the  solicitation  from  receiving  award  even  if  he  was  the  offeror  with  the 
lowest  price. 

The  “Sealed  Bids  -  Multiple”  model  was  designed  through  modification  of  the 
“Sealed  Bid  -  Aggregate”  model.  The  model  initially  resembles  Figure  20  of  the 
aggregate  model  for  price  and  quantity,  except  the  formulas  “=CELL  ADDRESS”  were 
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inserted  into  the  fill-in  sections  to  prevent  the  contracting  official  from  re-entering  the 
offeror  price  and  quantity  required  a  second  time.  Figure  24  demonstrates  the  differences 
between  the  aggregate  and  multiple  awards  models.  The  multiple  award  model  had  a 
requirement  to  add  $500  per  contract  awarded  to  evaluate  if  the  additional  administrative 
costs  of  awarding  multiple  contracts  was  more  beneficial  to  the  Government.  Since  the 
$500  administrative  fee  was  only  for  evaluation,  it  was  not  added  as  a  constraint  within  the 
SOLVER  model.  This  separation  permitted  the  evaluator  to  compare  the  total  price 
including  the  administration  fee  with  the  total  price  calculated  in  the  aggregate  model  to 
determine  which  method  of  award  provided  the  lowest  overall  cost  to  the  Government.  If 
the  determination  was  that  multiple  awards  were  the  most  beneficial,  then  the  contracting 
official  could  use  the  answers  provided  from  the  SOLVER  program  to  differentiate  which 
offerors  would  receive  award,  which  CLINs  would  be  awarded  to  each  offeror,  and  the 
amount  of  each  offeror’s  total  award. 

4. 3. 1.1  Communication.  The  magnitude  of  the  spreadsheet  required  to 
incorporate  all  of  the  data  necessary  to  solve  the  problem,  made  communication  of  the 
data  difficult.  Reducing  the  size  of  the  spreadsheet  to  40-50%,  permitted  the  evaluator  to 
review  a  significant  portion  of  the  data,  however  the  text  became  quite  small.  Color  codes 
were  used  to  assist  with  easier  communication  of  data.  Cells  were  highlighted  in  red 
wherever  the  contracting  official  was  required  to  insert  data  about  the  acquisition  or  the 
offeror.  The  changing  cells,  the  quantities  and  the  binary  constraint  cells,  were  a  dark 
grey;  and  the  target  cell,  or  total  price,  where  SOLVER  provides  the  final  solution  was 
dark  blue.  The  total  price  in  the  “Quantity  Offered  Per  CLIN”  matrix  was  also  highlighted 
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light  blue  to  indicate  which  offeror  would  receive  award  and  the  amount  of  the  award. 
Yellow  “reminder”  cells  were  added  beside  cells  requiring  constraints  to  be  input  into  the 
SOLVER  program  (Figure  22).  Light  green  cells  were  incorporated  to  provide 
information  on  specific  formulas  used  within  a  group  of  cells  (Figure  22).  These  are  just 
some  of  the  ways  colors  were  used  to  communicate  information  to  the  contracting  official. 

The  layout  of  the  spreadsheet  facilitated  communication  regarding  the  solution 
through  placement  of  information  cells  at  the  beginning  of  the  spreadsheet  (Figure  20  and 
Figure  22).  The  total  price  cell  (Figure  22,  dark  blue)  provided  the  solution  generated  by 
the  SOLVER  program  and  reflected  the  minimum  aggregate  price  to  the  Government. 

The  total  price  cells  (Figure  20,  light  blue)  provided  the  amount  each  offeror  would 
receive  for  their  individual  award  or  a  zero  indicating  no  award.  The  SOLVER  computed 
the  quantities  each  offeror  was  to  be  awarded  and  included  the  amount  in  the  changing 
cells.  In  the  aggregate  model,  only  one  offeror  could  be  selected  so  all  of  the  quantities 
for  each  CLIN  were  constrained  to  a  single  column.  The  “Total  QTY”  column  (Figure 
22)  indicated  whether  or  not  the  SOLVER  solution  met  the  Government’s  required 
quantity  levels  input  by  the  contracting  official.  The  SOLVER  replaced  the  changing  cells 
in  the  binary  matrix  to  either  “1”  or  “0,”  dependent  upon  whether  the  offeror  had  received 
an  award  or  not  (Figure  21).  The  “Binary:  Contractor  Used”  row  showed  which  offeror 
had  met  all  of  the  constraints  and  should  receive  award.  The  SOLVER  program  inserted  a 
“1”  in  the  cell  for  the  offeror  who  had  the  lowest  price  and  met  all  of  the  constraints.  All 
of  the  data  necessary  to  determine  which  offeror  should  receive  award  in  at  the  beginning 
of  the  spreadsheet,  except  for  verification  of  the  binaiy  variable  summary  for  the 
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constraint  cells.  (Figure  25,  constraint  summary  is  the  same  for  both  models).  Since  this 
binary  summary  was  used  only  for  verification,  placement  after  all  constraints  seemed 
plausible. 

4.3. 1.2  Reliability.  The  reliability  of  the  SOLVER  program  to 
consistently  and  accurately  provide  the  correct  solution  was  in  question.  Manipulation  of 
the  price  and  constraint  data  input  into  the  spreadsheet  revealed  some  problems  with  the 
SOLVER  programs  ability  to  perform  accurately  using  modified  data  once  the  SOLVER 
program  has  been  run.  At  one  point  during  analysis,  the  SOLVER  program  had  provided 
a  correct  solution  (based  upon  manual  calculations  for  verification);  but  continued  to 
provide  the  same  solution  when  the  pricing  data  for  another  offeror,  who  met  all  of  the 
constraint  requirements,  was  manipulated  to  a  very  obvious  lower  price  solution  when 
calculated  manually.  Regardless  of  how  the  data  was  manipulated,  the  SOLVER  program 
continued  to  generate  the  solution  it  had  originally  obtained.  Later  examination  revealed 
the  SOLVER  was  operating  correctly.  No  determination  could  be  made  as  to  the  cause  of 
this  problem,  whether  it  was  a  gliche  in  the  Windows  97  program  or  some  other  cause.  It 
could  have  resulted  from  SOLVER’S  ability  to  manipulate  the  data  as  it  was  formulated  in 
the  spreadsheet  models,  but  this  seems  an  unlikely  cause.  One  method  to  counteract  this 
problem  would  seem  to  be  reconstruction  of  a  new  model  for  each  new  acquisition;  rather 
than  modifying  the  model,  however  this  is  very  time-consuming  and  inefficient.  For  the 
benefit  received  from  the  development  of  this  model,  a  new  reconstruction  does  not  seem 
feasible  for  each  new  sealed  bidding  acquisition  determined  to  be  an  aggregate  award.  If  a 
computer  could  be  developed  to  design  the  models  for  the  contracting  official,  with  more 
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accuracy  in  the  solution,  it  might  become  feasible.  If  multiple  awards  were  authorized,  the 
ability  to  compare  the  results  of  the  two  models  might  make  the  reconstruction  feasible. 
Further  research  is  required  to  determine  the  cause  of  the  model’s  gliche  before  the  model 
can  be  considered  reliable. 

Another  reliability  consideration  resulted  from  modification  of  the  model.  Errors 
were  easily  made  when  using  the  copy/paste/edit  method  of  recreating  a  second  model. 

The  second  model  had  to  be  pasted  into  the  same  rows  and  columns  as  the  first  model 
(different  worksheet)  to  permit  stringed  cell  addresses  to  locate  the  correct  answer.  If  the 
second  model  was  copied  into  a  new  spreadsheet  document,  it  became  almost  mandatory 
to  verify  the  accuracy  of  the  formulas  in  every  cell. 

4.3. 1.3  Auditability.  The  ability  to  track  information  throughout  the 
document  is  difficult  due  to  its  enormous  size.  Copying  and  pasting  formulas  from  one 
cell  to  another,  placement  of  reminder  cells  within  the  spreadsheet  to  add  constraints  in  the 
SOLVER  program,  and  color  coding  cells  were  some  of  the  methods  employed  to  assist 
with  the  tracking  of  information  throughout  the  spreadsheet. 

The  use  of  stringed  address  cells  (i.e.  $L$4)  during  the  development  of  the  model 
prevented  data  from  being  changed  as  it  was  copied  from  one  cell  to  another.  This 
permitted  easy  verification  the  data  in  the  cells  were  accurate.  However,  the  use  of 
stringed  address  cells  caused  minor  problems  for  the  creation  of  new  models  to  be 
developed  based  upon  the  copy/past  e/edit  method. 

4.3. 1.4  Modifiability.  The  sealed  bid-aggregate  model  was  designed  and 
created  initially  and  used  as  the  basis  to  develop  the  sealed  bid-multiple  and  the 
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competitive  proposal-FAR  Part  15  models.  As  previously  stated,  the  use  of  stringed 
address  cells  caused  minor  problems  and  it  became  mandatory  to  verify  the  formulas 
copied  in  every  cell  to  ensure  the  formulas  were  correct.  Knowing  the  exact  solution  to 
the  problem  made  it  easy  to  know  when  the  model  was  accurately  solving  the  problem; 
however,  modification  was  time-consuming  as  several  iterations  of  the  SOLVER  program 
had  to  be  run,  and  minor  errors  corrected,  before  the  correct  answer  could  be  obtained. 

4. 3. 1. 5  Model  Limitations.  The  generation  of  an  evaluation  model  for  the 
assessment  of  offerors  proposal  information  was  successful;  however,  the  reliability  of  the 
Microsoft  Excel  SOLVER  Program  to  consistently  generate  accurate  solutions  in  these 
models  was  questioned.  This  inconsistency  could  be  the  result  of  the  Windows  97 
SOLVER  software  or  due  to  other  causes,  such  as  SOLVER’S  ability  to  manipulate  the 
data  in  this  particular  model.  This  accuracy  factor  was  seen  as  a  major  limitation  in  the 
use  of  the  Microsoft  Excel  SOLVER  program  for  decision-making  in  source  selection. 
SSAs  require  accurate  information  100%  of  the  time  and  it  has  not  been  determined  how 
reliable  the  SOLVER  model  is  at  total  accuracy.  Further  investigation  is  required  to  make 
that  determination. 

4.3.2  AFFARS  Appendix  AA/BB  Procedures.  The  AHP  process  was  used  to 
design  the  model  for  the  AFFARS  Appendix  AA/BB  source  selection  process.  One 
problem  encountered  during  development  was  devising  a  method  to  assign  preferences 
within  the  pairwise  matrix  without  comparing  an  offeror’s  proposal  against  its 
competitors.  To  alleviate  this  problem,  the  mandatory  AF  color  coding  rating  system  was 
used  to  evaluate  each  offeror  against  the  standard  for  each  criteria.  Once  a  color  code 
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was  assigned  to  the  criteria  at  the  element  level,  the  data  was  input  into  the  model  for  each 
offeror.  A  standard  numerical  rating  system  was  then  symbolically  applied  to  the  color 
codes  to  convert  the  qualitative  data  into  a  quantitative  means  of  evaluation. 

Windows  95  was  used  for  model  design  and  evaluation  of  the  AHP  model.  Design 
of  the  model  began  after  a  thorough  literature  review  of  the  acquisition  regulations  and 
examining  several  contract  files  to  observe  the  level  of  information  required  by  the  SSA  to 
make  an  effective  decision.  Based  upon  the  results  of  this  research,  the  model  was 
constructed  to  begin  input  of  information  at  the  element  level  of  evaluation.  A 
hypothetical  acquisition  was  devised  in  which  the  source  selection  participants  included 
three  offerors  (1,2,  and  3),  four  technical  evaluators  at  the  element  level  for  each 
subfactor,  a  factor/subfactor  team  chief  for  each  factor  (4),  a  cost/price  evaluator,  a  past 
performance  evaluator,  and  the  SSA.  Multiple  worksheets  were  developed  to  segregate 
various  types  of  data  at  the  different  evaluation  levels  (factor,  subfactor,  element).  The 
worksheets  in  the  model  start  at  the  highest  level  of  information,  and  subsequently  furnish 
detailed  data  at  the  lower  levels  in  each  worksheet  beneath.  The  “Final  AHP  Evaluation” 
worksheet  demonstrates  the  optimal  solution  to  the  problem  by  indicating  the  highest 
rated  offeror  as  the  best  alternative  choice.  The  “Criteria  Weights”  worksheet  compares 
the  preferences  or  ranking  of  the  factor  criterion  “Technical,  Cost/Price,  and  Past 
Performance”  in  relation  to  each  other  and  calculates  a  weighted  score  for  each  factor 
using  the  pairwise  matrix.  The  “Technical  Factors”  worksheet  takes  the  data  from  the 
“Technical  Factor  Ratings”  worksheet  to  calculate  AHP  scores  for  each  factor  using  the 
pairwise  matrix.  The  “Evaluator  Elements  Ratings”  worksheet  provides  the  input  into  the 
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model  from  the  individual  technical  evaluators  at  the  element  level  to  permit  the  team  chief 
to  subjectively  convert  the  data  into  a  subfactor  rating.  The  “Cost/Price”  and  “Past 
Performance”  worksheets  use  color  coded  input  from  the  source  selection  team  to 
determine  AHP  scores  using  the  pairwise  matrix.  Three  additional  worksheets  were 
provided  to  furnish  summary  data  at  the  element  and  subfactor  level.  These  worksheets, 
“Subfactor  Averages,”  “Technical  Element  Summary,”  and  “Technical  Elements”  were 
provided  for  informational  purposes  to  consolidate  and  communicate  the  detailed 
information  from  the  “Evaluator  Element  Ratings”  worksheet  should  the  SSA  require 
additional  information. 

Numerous  levels  of  subfactors  (4)  and  elements  (2-4)  were  generated  to  determine 
the  flexibility  and  difficulty  of  modifying  the  model  based  upon  minor  changes  which  could 
occur  in  different  acquisitions.  Initially  a  worksheet  “Evaluator  Element  Ratings”  was 
devised  to  input  the  color  codes  from  each  technical  evaluator  at  the  element  level  of 
criteria  evaluation.  Figure  9  demonstrates  the  level  of  input  provided  by  the  technical 
evaluator  at  the  element  level.  At  the  element  level  of  evaluation,  acquisition  regulations 
permit  the  use  of  either  color  coding  (red,  yellow,  green,  blue)  or  symbols  (plus  (+),  a 
check  (V),  and  minus  (-)).  It  should  be  noted  that  a  limitation  observed  within  the 
Microsoft  Excel  spreadsheet  program  was  the  inability  to  easily  place  a  check  (V)  symbol 
into  the  program.  Due  to  this  limitation,  color  coding  was  used  as  the  scoring  system  to 
input  evaluation  data  obtained  from  individual  technical  evaluators.  Once  the  data  was 
input  into  the  model,  the  factor/subfactor  team  chief  must  consolidate  the  information  at 
the  subfactor  level  and  provide  a  subfactor  rating.  A  second  limitation  which  was  noted 
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Figure  9.  Technical  Rating  Chart  of  a  Single  Evaluator,  and  Overall  Summary  of 


Subfactor  Ratings  Chart  Resulting  from  Consolidation  of  Technical 
Rating  Charts. 

with  the  AHP  model  was  its  inability  to  flow  model  scores  obtained  from  AHP  pairwise 
matrix  at  the  element  level  into  higher  levels  of  the  model  due  to  the  preference  criteria. 

At  each  new  level,  the  AHP  model  required  data  entry  of  the  “preferences”  based  upon  a 
rating  scale  required  at  that  level.  To  eliminate  this  problem  and  to  permit  the  team  chief 
to  observe  the  ratings  of  the  individual  evaluators  within  a  single  screen  setting,  a 
frequency  count  scoring  mechanism  was  derived  to  count  the  number  of  colors  per  offeror 
for  all  evaluators  and  provide  a  summary  total  to  the  team  chief  in  the  Overall  Rating 
table.  In  Figure  9,  the  color  codes  red  “R,”  yellow  “Y,”  green  “G,”  and  blue  “B,”  equate 
to  R  =  1000,  Y  =  100,  G  =  10,  and  B  =  1.  So  offeror  1  was  evaluated  for  Factor  1, 
subfactor  1,  element  1  as  having  zero  red,  zero  yellow,  one  green,  and  three  blue  (0013). 
The  team  chief  could  now  “roll-up”  the  subfactor  color  code  scores  into  the  next  level  of 
evaluation.  The  methodology  used  in  Figure  9  was  accomplished  for  each  subfactor  under 
all  factors. 

In  Figure  10(b),  the  formula  ‘-CELL  ADDRESS”  (i.e.  “-Evaluator  Element 
Ratings' !Z  14”)  was  entered  into  an  Element  cell  of  the  Overall  Subfactor  Rating  chart,  and 
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copied,  and  edited  accordingly  so  each  element  cell  would  automatically  be  updated 
should  the  evaluator  information  in  the  “Evaluator  Element  Ratings”  worksheet  be 
modified.  An  overall  subfactor  rating  chart  was  developed  for  each  subfactor.  Subfactor 
color  code  scores  were  assigned  a  numeric  symbol  (Figure  10(a))  to  standardize  the 
differences  in  the  scores  based  upon  preference.  Since  the  color  code  red  “R”  was  the 
least  desired  outcome,  indicating  an  unacceptable  rating,  it  was  assigned  a  symbol  of  “1;” 
yellow  “Y”  indicates  a  marginal  rating  so  it  was  assigned  a  symbol  of  “5;”  green  “G” 
indicates  an  acceptable  rating  so  it  was  assigned  a  symbol  of  “8;”  and  blue  “B”  indicates 
an  exceptional  rating  so  it  was  assigned  a  symbol  of  “  10.”  The  logic  for  these  rating 
symbols  was  to  permit  subtraction  of  the  numbers  so  the  difference  between  color  code 
ratings  for  each  offeror  could  be  equitably  compared.  The  formula: 

=IF(E10="R",$L$2,IF(E10="Y",$L$3,IF(E10="G",$L$4,IF(E10="B",$L$5))) 
was  entered  into  the  spreadsheet  cell  below  each  element  rating  color  code  so  the 
computer  would  automatically  translate  the  color  code  into  a  numeric  symbol.  Translation 
of  the  formula  means,  “If  the  color  code  in  cell  E10  is  R,  then  input  a  1;  if  the  color  code 
in  cell  E10  is  Y,  then  input  a  5;  if  the  color  code  in  cell  E10  is  G,  then  input  an  8;  and  if 
the  color  code  in  cell  E10  is  B,  then  input  a  10.”  Stringed  cell  addresses  (i.e.  $L$2)  were 
used  in  the  formula  rather  than  actual  numeric  symbols  (1,  5,  8,  10)  to  permit  flexibility  in 
the  model  should  it  become  necessary  to  modify  the  numeric  symbols,  and  to  allow  the 
formulas  to  be  copied  without  changing  to  different  cells.  The  average  of  each  offerors 
scores  was  calculated  to  determine  an  average  score  for  the  subfactor  using  the  formula 
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Figure  10.  Color  Code  Rating  Scale  and  Overall 
Subfactor  Rating  Chart. 

‘  - AVERAGE(E  1 1,E13)”,  where  the  cell  addresses  (El  1  through  E13)  are  referenced  for 
the  range  rather  than  using  actual  numbers  (10,  8)  to  permit  easy  modification. 

In  Figure  11,  factor  scores  were  generated  by  calculating  the  average  of  the 
subfactor  scores  for  each  offeror  again  using  the  “=AVERAGE(R12:R18)”  formula. 

Once  a  factor  average  score  was  derived,  it  was  then  converted  back  to  a  factor  level 
color  code  using  the  formula: 

=IF($M$2-R20>0,"R",IF($M$3-R20>0,"Y",IF($M$4-R20>0,"G","B"))) 
where  the  MIN/MAX  range  in  Figure  10(a)  was  used  to  determine  the  high  and  low  “cut 
off’  points  for  each  color.  $M$2,  $M$3,  and  $M$4  were  cell  addresses  for  the  maximum 
number  in  the  range  for  the  colors  red,  yellow,  and  green,  respectively.  The  formula 
stated  if  the  maximum  number  in  the  cell  subtracted  from  the  factor  average  (R20)  was 
greater  than  0,  then  input  that  color  code  (R,  Y,  or  G),  otherwise  the  computer  would 
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input  a  blue  “B”  rating.  The  range  for  the  color  code  red  “R”  was  0-5;  the  range  for  the 
color  code  yellow  “Y”  was  5.01  to  6.49;  the  range  for  the  color  code  green  “G”  was  6.50 
to  8.99;  and  the  range  for  the  color  code  blue  “B”  was  9-10.  The  logic  for  this  rating 
scale  range  was  to  permit  some  flexibility  in  rounding  the  average  factor  scores  to 
accommodate  for  the  loss  of  integrity  of  the  data  assumed  when  calculating  an  average 
(factor  level)  from  an  average  (subfactor  level). 

To  ensure  the  requirement  mandated  by  acquisition  regulations  which  stated  if  a 
rating  of  red  was  received  at  any  lower  level  then  it  must  be  shown  at  the  next  highest 
level,  a  formula  to  count  the  number  of  red  ratings  each  offeror  received  was  incorporated 
into  Figure  11.  The  formula  “=COUNTIF($E$10:$E$5 1,"R"),”  which  translates  “count 
one  if  a  cell  in  the  specified  range  is  equal  to  ‘R’.”  This  would  permit  the  team  chief  and 
SSA  to  know  if  an  offeror  had  received  any  red  ratings  at  a  lower  level  without  requiring 
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them  to  search  through  all  of  the  data  in  the  worksheets.  This  portion  of  the  “Average 
Score  Chart  for  the  Factors”  was  copied  to  the  “Technical  Factors”  and  “Final  AHP 
Evaluation”  worksheets  to  alert  evaluators  at  each  level  that  an  offeror  had  a  red  rating  at 
a  lower  level  and  could  not  receive  award.  A  separate  row  was  copied  to  the  “Final  AHP 
Evaluation”  worksheet  from  the  “Past  Performance”  worksheet  to  indicate  the  number  of 
red  ratings  received  on  past  performance. 

Figure  12  is  a  chart  from  the  “Technical  Factors”  worksheet  demonstrating  an 
AHP  pairwise  and  a  normalized  matrix.  An  AHP  matrix  was  constructed  for  each 
technical  factor  to  determine  a  factor  score  for  each  offeror.  These  factor  scores  were 
used  in  the  final  AHP  model  to  derive  the  overall  best  value  offeror.  To  avoid  re-input  of 
the  data,  the  factor  ratings  were  automatically  updated  from  the  “Technical  Factor 
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Figure  12.  Technical  Factor  Pairwise  and  Normalized  Matrices  in  an  AHP  Chart. 
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Ratings”  worksheet  using  the  formula  “=  CELL  ADDRESS”  ( i.e.  “—'Technical  Factor 
Ratings' !R21”)  and  to  calculate  the  value  of  the  color  code  the  formula: 

=IF(E9="R",$K$2,IF(E9="Y",$K$3,IF(E9="G",$K$4,IF(E9="B",$K$5)))) 

This  was  the  same  technique  as  applied  in  Figure  10  to  determine  the  values  of  the  color 
codes.  For  pairwise  comparisons,  the  formulas  in  the  matrix  to  input  the  preferences 
between  offerors  was: 

=IF  ((E 1 0-E 1 4)=0, 1  ,IF((E  1 0-E 1 4)>0,(E  1 0-E 1 4),(  1  /(E 1 4-E 1 0)))) 
which  stated  if  the  value  in  the  cell  for  offeror  1  minus  the  value  in  the  cell  for  offeror  2 
equals  0,  then  input  a  1  (equal  preferences);  if  the  value  in  the  cell  for  offeror  1  minus  the 
value  in  the  cell  for  offeror  2  is  greater  than  0,  then  input  the  difference;  otherwise,  input 
the  reciprocal  of  the  value  in  the  cell  for  offeror  1  minus  the  value  in  the  cell  for  offeror  2. 
The  reciprocal  formula  “=1/I8”  was  input  using  the  cell  address  into  the  cell  for  comparing 
offeror  2  to  offeror  1 .  These  formulas  were  copied  to  the  appropriate  cells  for  comparison 
of  all  offerors  in  each  of  the  four  factor  matrices.  The  formulas  for  the  normalized 
matrices  discussed  by  Ragsdale  (1997)  were  input  (using  cell  addresses  for  easy  editing) 
into  the  matrices  to  determine  the  factor  scores  and  consistency  ratios.  These  formulas 
include: 

=SUM(G8:G10)  (Sum  of  Offeror  Columns  in  Pairwise  Matrix) 

=G8/G$1 1  (Normalized  Matrix  -  i.e.  first  cell  in  column  of  pairwise  matrix  divided 

by  the  sum  of  the  column) 

=AVERAGE(G18:I18)  (Factor  Score  -  i.e.  average  of  the  cells  in  the  first  row  of 

the  normalized  matrix) 

=MMULT(G8:I8,$J$18:$J$20)/J18  (Consistency  Measure) 
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=( AVERAGE(K1 8 :K20)-3)/2/H22  (Consistency  Ratio,  where  H22  equals  the  RI 
factor) 

The  RI  factor  used  in  calculating  the  consistency  ratio  was  added  using  a  cell 
address  (H22)  as  a  reminder  to  personnel  that  this  factor  may  need  to  be  changed  based 
upon  the  number  of  “n,”  or  offerors,  being  compared.  Ragsdale  provides  a  RI  factor  scale 
with  a  range  from  2  -  8  (Table  7).  In  large  source  selections,  for  systems  acquisitions,  a 
maximum  of  n=8  would  appear  to  suffice  as  it  seemed  apparent  the  number  of  offerors 
was  generally  less  than  8. 
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Figure  13.  Past  Performance  Pairwise  and  Normalized  Matrices. 


To  evaluate  Past  Performance,  the  same  color  coding  concept  was  applied  as  used 
for  developing  the  technical  factor  scores.  Figure  13  demonstrates  the  application  of  the 
AHP  pairwise  and  normalized  matrices  to  rate  past  performance.  The  Past  Performance 
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scores  were  automatically  entered  into  the  “Final  AHP  Evaluation”  worksheet  for  final 
analysis. 

The  cost/price  criterion  (Figure  14)  was  rated  using  the  9-point  preference  scale 
from  Table  6  (Ragsdale,  1997).  The  preferences  are  subjectively  categorized  based  upon 
the  ranking  of  the  offers.  In  this  model,  it  was  assumed  that  offeror  1  had  the  highest 
price,  $3,000,000;  offeror  2  had  the  lowest  price  $2,500,000;  and  offeror  3  was  in  the 
middle  range  with  an  offer  of  $2,800,000.  Since  the  difference  between  the  highest  and 
lowest  offers  was  not  extremely  significant,  a  rating  of  7-  Very  Strongly  Preferred  was 
assigned  to  compare  the  difference  between  offerors  2  (low)  and  1  (high)  because  of  the 
$500,000  difference  in  price.  A  rating  of  5-  Strongly  Preferred  was  assigned  to  compare 
the  difference  between  offerors  2  (low)  and  3  (medium)  because  of  a  $300,000  difference 
in  price.  A  rating  of  3-  Moderately  Preferred  was  assigned  to  compare  the  difference 
between  offerors  3  (medium)  and  1  (high)  because  of  a  $200,000  difference  in  price. 

Since  the  reciprocal  formulas  were  entered  into  the  chart’s  cells  where  the  row  (offeror  2) 
intersects  with  column  (offeror  1);  row  (offeror  3)  intersects  with  column  (offeror  1);  and 
row  (offeror  3)  intersects  with  column  (offeror  2),  the  reciprocal  formulas  “==1/7”  and 
“=l/3”  had  to  be  manually  entered  into  the  cells  where  row  (offeror  1)  intersects  with 
column  (offeror  2)  and  row  (offeror  1)  intersects  with  column  (offeror  3),  respectively. 
This  was  necessary  because  the  prices  for  offerors  2  and  3  were  higher  than  offeror  1,  in 
the  comparisons  of  offerors  1  to  2  and  1  to  3.  The  number  “5”  was  manually  input  into 
the  row  where  (offeror  2)  intersects  with  column  (offeror  3).  The  reciprocal  was  not 
required  in  this  cell  because  the  price  for  offeror  2  was  lower  than  the  price  for  offeror  3. 
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The  Cost/Price  Scores  in  the  normalized  matrix  were  calculated  using  the  formulas  for  the 
pairwise  and  normalized  matrices  given  previously  (paragraph  2.4. 6. 2). 
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Figure  14.  Price/Cost  Pairwise  and  Normalized  Matrices. 


Figure  15  provides  a  ranking  scale  for  each  factor  criterion  in  relation  to  the  other 
criterion.  FAR  1 5-3 04(e)  states  “the  solicitation  shall. . .  state,  at  a  minimum,  whether  all 
evaluation  factors  other  than  cost  or  price,  when  combined,  are- 

(1)  Significantly  more  important  than  cost  or  price; 

(2)  Approximately  equal  to  cost  or  price;  or 

(3)  Significantly  less  important  than  cost  or  price”  (FAR  15-304(e)) 

This  criterion  weight  matrix  permitted  the  criterion  to  be  ranked  in  order  of 
importance  based  upon  a  9-point  preference  scale  (Figure  1 5)  which  was  similar  to 
Ragsdales  (Table  6).  In  Figure  16,  Factor  1  was  the  most  important  factor  with  Factors  2, 
3,  and  4  less  important  and  following  in  descending  order  of  importance.  In  relation  to 
Factor  1,  rated  as  1,  these  factors  were  rated  as  “2,”  “3,”  and  “4,”  respectively.  Price  and 
Past  Performance  were  of  equal  importance  but  were  less  important  than  the  technical 
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Figure  15.  Criterion  Weights  9-Point  Preference  Scale  for 


Acquisition  Model. 

factors  so  they  were  both  ranked  as  “5’s”.  Review  of  the  normalization  matrix  and 
consistency  ratio  in  Figure  16  indicated  that  the  ranking  procedure  was  an  acceptable 
methodology.  The  Criterion  Weights  in  the  normalized  matrix  were  calculated  using  the 
formulas  for  AHP  pairwise  and  normalized  matrices  (paragraph  2. 4. 6.2).  An  RI  factor  of 
1 .24  was  used  because  the  number  of  offerors  equated  to  n=6  criterion.  The  formula  in 
the  cell  calculating  the  consistency  ratio  was  modified  as: 

=((  A  VERAGE(L  1 6 :  L2 1  )-6)/5)/I22 

to  account  for  the  increase  in  the  number  of  “n.”  The  criterion  weights  for  each 
factor/criterion  were  automatically  entered  into  the  “Final  AHP  Worksheet”  for 
calculation  of  the  weighted  average  scores  of  each  offeror. 

Figure  17  provides  the  summary  data  used  for  presentation  to  the  SSA.  The 
scores  calculated  from  the  AHP  pairwise  and  normalized  matrices  for  each  Technical 
Factor,  Cost/Price  and  Past  Performance  criterion  were  automatically  entered  into  the 
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Figure  16.  Criteria  Weight  Scores  Pairwise  and  Normalized  Matrices. 


AHP  final  matrix  using  the  “=CELL  ADDRESS”  formula.  Using  the  formula: 

=SUMPRODUCT(E7:E  1 2,$H$7 :  $H$  1 2) 

a  weighted  average  score  was  calculated  for  each  offeror.  This  formula  multiplied  the 
factor/criterion  score  for  each  offeror  by  its  respective  criterion  weight,  summed  the 
products  and  computed  a  weighted  average.  The  offeror  with  the  highest  weighted 
average  was  considered  the  offeror  with  the  optimal  solution  and  should  have  been 
recommended  to  receive  award.  However,  before  award  could  be  made,  the  source 
selection  team  needed  to  ensure  there  were  no  deficiencies  contained  within  the  proposal 
with  the  highest  score.  A  chart  to  show  the  number  of  red  ratings  for  both  technical 
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factors  and  past  performance  was  included  on  the  final  AHP  evaluation  chart  to  indicate  if 
detailed  investigation  was  necessary  by  the  source  selection  team.  For  offerors  with 
weighted  average  scores  which  are  fairly  close  in  range,  trade-off  analysis  should  be 
conducted  to  ensure  the  highest  score  reflects  the  absolute  best  value  alternative.  The 
concept  of  an  AHP  model  for  use  in  the  source  selection  decision-making  process  appears 
viable.  The  next  section  discusses  the  model  quality  in  relation  to  communication, 
reliability,  auditability,  and  modifiability. 
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Figure  17.  Final  AHP  Scoring  Chart. 


4. 3.2.1  Communication.  The  AFFARS  Appendix  AA/BB  AHP  model 
was  designed  using  multiple  worksheets  to  furnish  top  level  information  required  by  a  SSA 
to  make  a  decision  in  the  first  worksheet.  The  “Final  AHP  Model”  worksheet  provides  a 
factor  level  summary,  and  subsequent  worksheets  expand  the  data  evaluated  into  lower 
level  subfactors  and  elements  to  provide  detailed  information.  At  the  element  level. 
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additional  worksheets  (“Subfactor  Averages,”  “Technical  Elements  Summary,”  and 
“Technical  Elements”)  were  provided  to  demonstrate  the  differences  between  offerors  for 
each  technical  subfactor  at  the  element  level.  This  information  did  not  flow  into  the  AHP 
model  but  was  designed  to  furnish  detailed  information  should  the  SSA  request  it. 

In  addition,  in  the  “Evaluator  Element  Rating”  worksheet,  a  frequency  count 
formula  was  devised  in  the  overall  summary  matrix  to  permit  the  team  chief  to  compare 
the  number  of  each  color  attributed  to  each  element  at  a  glance.  The  values  of  1000,  100, 
10,  and  1  were  assigned,  respectively,  to  the  colors  red,  yellow,  green,  and  blue.  If  four 
evaluators  collectively  assigned  2  blue,  1  green,  and  1  yellow  to  an  offeror  for  a  specific 
element,  then  the  frequency  count  formula  would  calculate  and  assign  a  value  of  01 12. 

This  communicates  to  the  team  leader  on  the  consistency  of  the  ratings  between  evaluators 
and  permits  him/her  to  investigate  potential  discrepancies  and/or  assign  a  subfactor  rating 
based  upon  the  individual  elements  scores. 

Another  communications  tool  was  the  consistent  use  of  colors  throughout  the 
model  to  signify  specific  types  of  data.  For  example,  award  cannot  be  made  to  any  offeror 
who  has  a  deficiency  at  a  lower  level.  To  provide  an  audit  trail,  and  to  communicate  to 
the  SSA  that  an  offeror’s  proposal  had  a  deficiency,  or  “red”  rating,  a  count  of  the  number 
of  red  ratings  was  provided  in  red-colored  cells  which  were  placed  below  the  overall 
ratings  provided  to  the  SSA.  This  would  direct  the  attention  of  the  SSA  to  the  red  cells  in 
the  spreadsheet  to  inform  him/her  if  any  red  cells  contained  a  number  >  0,  then  award 
could  not  be  made  to  that  offeror. 
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43.2.2  Reliability.  The  consistency  factors  in  the  AHP  model  provided  a 
certain  level  of  reliability  to  the  model.  The  major  problem  observed  with  the  AHP  model 
was  in  the  use  of  the  rating  scale  preferences  provided  by  Ragsdale  in  Table  6.  The 
consistency  ratio  was  often  >0.10  when  the  preferences  were  at  the  higher  end  of  the 
rating  scale.  This  was  noted  mainly  in  the  ranking  of  criterion  in  the  “Criteria  Weights” 
worksheet.  If  the  preferences  were  subjectively  assigned  in  the  cells  based  on  preferences, 
the  chance  for  error  increased  as  the  consistency  scores  became  extremely  high.  Ranking 
the  technical  factors,  for  example  from  1-4,  cost/price  at  5,  and  past  performance  either  5 
(equal  with  cost/price)  or  6  (lower  than  cost/price),  and  continuing  this  methodology 
throughout  the  matrix  resulted  in  consistency  ratios  <  0. 10  and  reduced  the  chance  for 
error  (Figure  16).  To  prevent  errors  from  occurring  at  the  element  level  and  past 
performance  ratings,  a  new  standard  rating  scale  was  devised  for  the  color  codes.  A  “1” 
was  assigned  to  all  red  ratings,  a  “5”  was  assigned  to  all  yellow  ratings,  an  “8”  was 
assigned  to  all  green  ratings,  and  a  “10”  was  assigned  to  all  blue  ratings.  Once  these  new 
ratings  were  applied  to  the  model,  the  consistency  ratio  normalized  itself  and 
systematically  measured  <  0.10  and  most  often  0.00. 

For  offerors  with  weighted  average  scores  which  are  reasonably  close  in  range, 
trade-off  analysis  should  be  conducted  to  ensure  the  highest  score  reflects  the  absolute 
best  value  alternative.  Since  the  Technical  Factor  Scores  are  based  upon  averages  of 
lower  level  criterion  averages,  some  integrity  of  the  rating  data  could  be  lost  during  the 
“roll-up”  of  averages  Due  to  this  fact,  the  proposal’s  of  offerors  that  have  final  weighted 
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scores  fairly  proximal  should  be  impartially  reviewed  to  ensure  all  determinants  have  been 
fairly  evaluated  and  the  Government  is  receiving  the  best  value  alternative. 

43.2.3  Auditability.  Color  coding  was  used  in  the  model  to  assist 
personnel  trying  to  audit  the  results  of  the  model  to  understand  and  verify  what  level  of 
information  was  being  obtained.  Factor  information  was  turquoise,  subfactor  information 
was  pink,  element  information  was  light  blue.  Data  obtained  from  AHP  Scores  were 
yellow,  data  from  averaging  information  was  light  green,  data  indicating  the  maximum  or 
highest  number  within  a  range  was  light  purple,  and  rating  scales  were  light  yellow.  These 
were  just  a  few  examples  where  colors  were  used  to  permit  user  auditability. 

Formulas  (=AVERAGE('Technical  Element  Summary'!F8:F9))  and  cell  references 
(='Evaluator  Element  Ratings' !$Z$  14)  were  used  within  the  cell  addresses  rather  than  hard 
numbers  to  permit  the  efficient  flow  of  information  from  one  worksheet  to  another.  If 
information  changed  on  an  offerors  evaluation  at  the  element  level,  the  new  data  was  input 
and  the  computer  would  automatically  update  the  remainder  of  information  to  provide  the 
new  results  at  the  highest  level. 

43.2.4  Modifiability.  The  spreadsheet  was  designed  for  flexibility  in 
modifying  the  data  contained  in  the  baseline  AHP  model,  or  even  to  create  new  models 
through  the  copy  and  paste  feature  of  the  Microsoft  Excel  program.  Using  the  “insert 
cells”  function,  the  source  selection  team  is  permitted  to  insert  additional  offeror 
information  or  criterion  factors  into  the  AHP  matrix  without  harming  the  data  already 
contained  in  the  matrix.  The  AHP  matrices  were  aligned  in  the  spreadsheet  to  easily  add 
rows  or  columns  and  still  maintain  the  integrity  of  the  data.  Formulas  could  be  copied  to 
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the  cells  for  any  additional  offeror  from  existing  offeror  cells.  AHP  matrices  could  be 
copied  to  new  worksheets  with  minimal  editing,  and  no  editing  requirements  if  they  were 
placed  in  the  same  rows  and  columns  as  on  the  previous  worksheet.  New  factors, 
subfactors,  and  elements  could  be  added  in  the  same  fashion. 

The  deletion  of  offerors  from  the  matrix,  or  the  deletion  of  factors,  subfactors,  and 
elements  could  be  accomplished  using  the  “delete,  shift  cells. . .”  feature  of  the  program. 
This  feature  permits  information  to  be  deleted  but  still  maintains  the  integrity  of  the 
remaining  data. 

4.3. 2.5  Limitations  of  the  AHP  Model.  This  model  was  designed  with 
the  assumption  that  assessment  criteria,  general  considerations,  proposal  risk  and 
performance  risk  would  be  evaluated  either  prior  to  entering  color  codes  into  the  model  or 
after  the  Final  AHP  evaluation  was  completed.  No  provisions  were  built  into  the  model  to 
assess  these  criteria.  Excel  models  using  matrices  currently  exist  to  conduct  risk 
assessment  so  this  area  was  not  addressed  in  this  thesis  (Garvey  and  Lansdowne,  1998). 

Changes  in  proposal  evaluation  data  compiled  from  Best  and  Final  Offer  analysis 
could  be  easily  input  into  the  spreadsheet  so  the  final  AHP  evaluation  matrices  could  be 
compared  to  determine  changes  in  ratings  based  upon  reevaluation  of  proposals. 
Comparison  for  Best  and  Final  Offers  was  not  automatically  built  into  the  model.  The 
model  has  potential  for  trade-off  analysis,  however  changing  the  color  codes  at  the 
element  level  would  be  necessary  to  obtain  the  necessary  results.  The  AHP  model  may 
not  be  the  best  method  for  conducting  trade-off-analysis. 
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4.3.3  FAR  Part  15,  Competitive  Procedures.  The  Microsoft  Excel  SOLVER 
program  using  ILP  was  used  to  design  the  model  for  Competitive  acquisitions,  using 
procedures  under  FAR  Part  15.  Windows  97  was  used  in  the  development  of  this  model. 
Appendix  D,  Figures  25  -  27  provide  visual  data  of  charts  from  the  actual  model 
developed.  Appendix  D,  Table  9  provides  the  SOLVER  parameters  and  options  necessary 
to  solve  the  ILP  problem.  The  competitive  acquisition  model  was  designed  based  upon 
the  lowest  priced,  technically  acceptable  offeror  concept.  Like  the  sealed  bidding  model, 
the  model  was  arranged  to  permit  the  contracting  official  to  input  specific  background 
data,  and  to  receive  some  direction  from  the  initial  “Acquisition  Background”  worksheet. 

A  single  SOLVER  model  was  designed  to  permit  award  of  a  single,  or  “aggregate” 
contract  because  multiple  awards  were  unlikely.  The  “FAR  Part  15”  worksheet  was 
copied  and  modified  using  the  “Sealed  Bid  -  Aggregate”  worksheet.  Data  on  technical 
acceptability  was  initially  input  by  the  contracting  official  into  a  factor  rating  chart  (Figure 
26)  in  the  “Technically  Acceptable”  worksheet.  This  worksheet  was  copied  and  modified 
from  the  AFFARS  Appendix  AA/BB  source  selection  AHP  model.  Figure  27  diagrams 
the  rating  scale  and  values  used  to  determine  technical  acceptance  of  an  offeror.  An 
acceptable,  or  “A”  rating  means  the  proposal  is  accepted  as  submitted.  A  marginal,  or 
“M”  rating  means  the  proposal  was  marginal  but  could  reasonably  be  made  acceptable 
through  the  submission  of  clarifying  or  supplemental  information,  through  discussions, 
which  does  not  substantially  change  the  proposal  information  already  submitted.  An 
unacceptable,  or  “U”  rating  means  the  proposal  is  not  acceptable.  Using  binary 
constraints,  an  acceptable  rating  was  given  a  factor  of  “1”  and  a  marginal  and 
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unacceptable  rating  were  given  a  factor  of  “0.”  A  marginal  proposal  was  not  factored  as  a 
“1”  because  it  would  be  necessary  to  change  the  “M”  to  an  “A”  and  become  acceptable 
through  discussions  or  clarifications  before  it  could  receive  award.  The  SSA  would  be 
required  to  make  a  determination  whether  or  not  discussions  were  warranted  if  the 
provision  stating  award  can  be  made  without  discussions  was  included  in  the  solicitation. 
The  technical  data  was  input  at  the  element  level,  as  in  the  AHP  model,  but  was 
subjectively  rated  through  the  Factor  level  by  the  technical  team  chief.  The  technical 
factor  ratings  were  then  transferred  to  the  “FAR  Part  15”worksheet  using  the  “-CELL 
ADDRESS”  formula.  The  technical  data  was  incorporated  into  the  SOLVER  model 
through  the  use  of  the  binary  constraints  (Figure  25).  As  in  the  sealed  bid  model,  a 
worksheet  “Unbalanced  Bids”  using  graphics  was  provided  to  permit  analysis  of  offeror’s 
proposals  for  indications  of  unbalanced  proposals  or  front-end  loaded  proposals.  Fill-in 
data  from  the  “Unbalanced  Bids”  worksheet  would  be  transferred  to  the  “FAR  Part  1 5” 
worksheet. 

4.3.3. 1  FAR  Part  IS,  Model  Analysis.  Since  the  FAR  Part  1 5  model  was 
created  by  modifying  the  sealed  bid  model  for  the  SOLVER  input  and  the  AHP  model  for 
the  technical  data,  the  discussion  on  communication,  reliability,  auditability,  modifiability, 
and  model  limitations  were  the  same  as  in  the  sealed  bidding  and  in  some  of  the  AHP 
technical  portions  of  the  previous  sections.  This  model  would  have  the  same  reliability 
concerns  as  the  sealed  bid  model,  due  to  the  use  of  the  SOLVER  program.  However,  the 
technical  data  in  this  model  was  not  “rolled  up”  using  averages  as  in  the  AHP  pairwise 
model.  The  technical  factors  were  manually  evaluated  by  the  contracting  official  who  had 
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to  subjectively  assign  a  higher  rating  based  upon  the  information  provided  at  the  lower 
levels.  Pairwise  matrices  were  not  used  for  analysis  in  this  model. 

4.4  Conclusion 

Chapter  4  has  answered  the  investigative  questions  first  discussed  in  Chapter  1  of 
this  thesis.  A  schematic  model  of  information  technology  was  designed,  and  represents 
one  of  many  solutions,  incorporating  information  technology  into  the  source  selection 
decision-making  process.  Three  decision-making  models  were  designed  and  developed 
using  the  Microsoft  Excel  spreadsheets  program.  Two  of  the  models  (sealed  bids  and 
competitive  procedures,  FAR  Part  15)  used  the  SOLVER  option  program,  and  one 
(AFFARS  Appendix  AA/BB)  used  the  Analytical  Hierarchy  Process.  The  SOLVER 
program  was  not  used  on  the  AFFARS  Appendix  AA/BB  source  selection  model  since  the 
AHP  model  was  the  preferred  approach.  Due  to  the  enormous  size  of  the  models,  it  was 
impossible  to  provide  complete  copies  of  the  model  within  the  text  of  this  document.  As 
much  information  was  provided  to  permit  the  reader  to  visualize  the  basic  concepts  of  the 
models  and  to  understand  the  textual  discussions  within  this  document. 

Chapter  5  discusses  the  overall  summary  of  the  research  and  provides 
recommendations  based  upon  the  findings  summarized. 
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5.  Summary  and  Recommendations 


5.1  Introduction 

Decision-making  in  a  source  selection,  whether  it  be  a  large  scale  source  selection 
as  in  using  AFFARS  Appendix  AA/BB  procedures  or  a  smaller  scale  using  sealed  bidding 
or  competitive  procurement  procedures,  is  an  important  aspect  of  whether  the  AF  and  the 
Government  successfully  perform  their  mission.  The  increase  of  outsourcing  work  to 
contracted  sources  makes  good  decision-making  in  the  evaluation  of  offerors  imperative 
to  permit  the  AF  to  meet  its  requirements  within  its  budget  constraints.  The  information 
technology  systems  discussed  in  this  research  could  enhance  the  SSA’s  ability  to  make 
better  decisions  in  a  more  efficient  manner. 

5.2  Summary  Review  of  the  Research 

5.2.1  Investigative  Question  Number  One.  Can  a  schematic  model  be  designed 
to  identify  areas  where  information  technology  can  improve  the  flow  of  information  into 
the  source  selection  evaluation  process  and  expand  the  present  tactical  approach  to 
evaluation  of  alternatives  into  a  more  strategic  approach? 

5. 2. 1. 1  Findings  and  Recommendations. 

1 .  A  schematic  information  technology  model  was  designed  and  the  technology 
exists  to  realize  this  information  system.  This  information  system  has  the  potential  to 
transform  the  acquisition  community  into  a  more  strategic,  proactive  component  within 
the  AF  and  Government,  as  a  whole.  The  next  step  prior  to  development  and 
implementation  of  an  application  model  would  be  to  analyze  and  evaluate  application 
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models,  such  as  a  DSS  or  ES,  which  are  currently  operating  in  a  business  decision-making 
environment.  Evaluation  and  comparison  of  the  hardware,  software,  and  other 
information  technology  components  used  in  the  development  and  operation  of  these  types 
of  systems  prior  to  development  could  permit  a  more  efficient  and  effective  system  design 
to  be  implemented  within  the  acquisition  field.  Operational  analysis  of  systems  currently  in 
use  would  permit  contracting  officials  to  evaluate  the  functionality  of  a  DSS  system  and  its 
components  to  determine  if  the  schematic  model  designed  in  this  research  is  the  best 
solution.  Analysis  of  various  systems  in  operation  would  permit  a  comparison  of  system 
application  with  the  literature  to  determine  if  the  systems  could  actually  function  as  the 
literature  claims  or  if  modification  of  the  system  would  be  required. 

2.  The  Government  acquisition  field  is  unique  with  its  numerous  policies,  laws, 
and  regulations.  Decision  support  systems  developed  for  use  for  commercial  acquisitions 
would  not  contain  many  of  the  constraints  or  restrictions  placed  upon  the  Government 
within  their  systems.  Modification  of  these  systems  would  be  necessary  prior  to 
implementation  in  the  Government  acquisition  field.  Should  a  commercial  system  be 
selected  for  implementation,  it  is  recommended  it  be  reviewed  for  flexibility  and 
adaptability  to  incorporate  the  required  modifications  at  the  lowest  possible  cost.  This 
requirement  became  apparent  during  the  development  of  the  AHP  evaluation  model  for 
AFFARS  Appendix  AA/BB  source  selections.  Much  of  the  literature  reviewed 
recommended  the  use  of  AHP  techniques  for  the  design  and  development  of  a  source 
selection  model  because  it  could  incorporate  the  qualitative  data  resulting  from  evaluation 
of  technical  and  past  performance  criteria.  However,  actual  implementation  of  the  model 
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revealed  slight  modification  was  necessary  due  to  the  Government  requirement  which 
restricts  comparison  of  offerors  against  their  competitors. 

3 .  Depending  upon  the  type  of  model  base  implemented  in  the  schematic  model,  a 
better  user  interface  device  should  be  explored  to  permit  contracting  officials  to  easily 
develop  models  for  each  new  acquisition.  The  user  interface  device  should  permit  the 
automatic  generation  of  models  based  upon  a  specific  acquisition  situation.  It  should  also 
permit  the  automatic  generation  of  reports  and  sensitivity  analysis  to  determine  how 
changes  in  constraints  affect  the  results  of  the  selection.  Further  research  might  be 
conducted  in  the  use  of  Visual  Basic  for  Applications  (VBA)  (Hill,  1998)  and  how  it 
could  be  incorporated  into  the  SPS  system  to  permit  the  automatic  model  generation. 

5.2.2  Investigative  Question  Number  Two.  Can  a  baseline  model  be  designed, 
based  upon  Microsoft  Excel  SOLVER,  to  assist  the  SSA  in  choosing  the  optimal  or  best 
alternative  source  in  a  competitive  source  selection? 

5. 2. 2. 1  Findings  and  Recommendations. 

1 .  The  use  of  Management  Science  techniques,  such  as  ELP  and  AHP,  in  the 
design  and  development  of  a  decision-making  model  base  can  be  successfully 
accomplished  using  spreadsheet  design.  Three  evaluation  models  were  developed  using 
ILP  or  AHP  techniques  which  accurately  provided  a  best  alternative  solution  based  upon 
the  criteria  and  constraints  for  the  problem.  However,  verification  of  the  models  was 
conducted  on  a  very  limited  basis.  Further  analysis  using  test  cases  of  actual  contract  data 
should  be  conducted  to  validate  the  use  of  the  spreadsheet-based  models  to  evaluate 
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acquisition  data.  The  model’s  ability  to  generate  accurate  and  reliable  solutions 
consistently  needs  to  be  assessed  and  validated  prior  to  implementation. 

2.  One  problem  noted  during  development  resulted  from  consistent  reliability 
when  using  the  ILP  models  utilizing  the  Microsoft  Excel  SOLVER  program.  The  ILP 
models  performed  as  expected  and  initially  solved  the  problems  correctly,  but  an 
occasional  loss  of  reliability  occurred  when  minor  changes,  such  as  offered  price,  were 
implemented  in  the  problem.  This  loss  led  to  a  concern  regarding  the  reliability  of 
solutions  generated  in  the  SOLVER  program,  and  more  so  when  the  acquisition  problem 
became  so  extensive  that  manual  verification  could  not  be  easily  accomplished. 

Acquisition  data,  such  as  prices  and  technical  constraints,  are  often  modified  based  upon 
discussions  during  a  source  selection  and  the  contracting  official  must  ensure  the 
evaluation  method  used,  whether  a  computer  or  manual  manipulation,  can  reliably 
accommodate  and  adapt  for  these  changes.  Two  recommendations  for  further 
investigation  are  recommended  as  a  result  of  the  reliability  problem.  First,  examination  of 
the  model’s  formulation  to  determine  if  it  is  too  sensitive  for  use  by  Excel’s  SOLVER 
program.  This  sensitivity  could  result  in  the  occasional  unreliability  seen  during  the  initial 
development.  Second,  a  recommendation  to  explore  other  Frontline  SOLVER  and 
spreadsheet  model  software  programs  to  determine  if  other  software  has  more  reliable 
capabilities  to  incorporate  changes. 

3 .  The  AHP  model  was  reliable  in  consistently  generating  accurate  solutions  but 
the  averaging  of  average  evaluation  scores  from  lower  levels  of  evaluation  might  be 
considered  a  protestable  issue  to  an  offeror  losing  the  award  to  another  contractor.  To 
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deter  protests,  further  research  would  be  necessary  to  analyze  and  develop  standard 
coding  systems  for  rating  conversion  (i.e.  B  =  10,  G  =  8,  Y  =  5,  R  =  1)  for  use  in  the 
AHP  model.  A  standard  coding  system  would  provide  consistency  among  contracting 
officials  permit  a  more  objective  evaluation  because  the  numeric  standard  rating  for  a 
qualitative  color  or  adjectival  ratings  would  remain  unchanged  from  acquisition  to 
acquisition. 

4.  The  extent  of  training  and  educating  personnel  in  the  acquisition  field  to  use  the 
management  science  tools  for  optimization  in  decision-making,  the  use,  capabilities,  and 
limitations  of  programs  such  as  Microsoft  Excel  SOLVER  and  the  AHP  pairwise  matrices, 
and  to  comprehend  and  obtain  the  mathematical  (i.e.  linear  functions  and  matrices)  and 
computer  (spreadsheet  capabilities)  could  be  another  deterrent  from  implementation  of  the 
use  of  management  science  tools  in  the  evaluation  process.  These  skills  and  knowledge 
are  required  to  design  and  development  an  evaluation  model.  A  study  could  be  conducted 
to  examine  the  knowledge  and  skill  levels  of  contracting  professionals  and  a  determination 
could  be  made  on  the  amount  of  training  to  implement.  Another  recommendation  to  avoid 
extensive  training  of  personnel  in  advanced  mathematics  and  computer  knowledge  would 
be  to  include  Operations  Research  personnel  on  teams  to  design  the  information  systems 
for  the  procurement  field.  The  DoD  Standard  Procurement  System,  currently  in 
development,  could  be  designed  to  incorporate  user-friendly  evaluation  models  to  permit 
the  computer  to  develop  the  models  based  upon  data  entered  and  responses  obtained  from 
contracting  officials.  A  user  friendly  interface  could  prevent  the  need  for  detailed  training. 
In  addition,  a  study  to  analyze  potential  constraints  contained  within  the  acquisition 
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regulations  could  be  accomplished  and  a  manual  developed  to  instruct  personnel  on  the 
procedure  to  input  these  constraints  into  the  optimization  program.  These  would  also 
reduce  the  amount  of  training  necessary  for  personnel  to  implement  management  science 
tools  in  the  evaluation  process  of  a  source  selection. 

5.  Different  types  of  acquisitions  require  the  use  of  different  types  of  evaluation 
tools.  A  sealed  bid  acquisition  restricted  to  an  aggregate  award,  with  a  small  number  of 
CLIN  and  offerors,  could  easily  be  evaluated  using  a  basic  spreadsheet  to  calculate  the 
total  price  of  each  offeror.  The  use  of  the  SOLVER  program  would  not  benefit  the 
evaluation  process  in  an  acquisition  of  this  type.  However,  for  larger  acquisitions  which 
permit  multiple  awards  or  have  numerous  constraints,  offerors,  and  CLINs,  the 
development  of  a  model  using  a  SOLVER  program  could  prove  beneficial.  The  AHP 
model  is  effective  in  determining  best  value  for  Appendix  BB  source  selection  but  other 
models,  such  as  an  optimization  model,  should  be  explored  as  a  viable  method  of  solving 
these  decision  problems.  The  main  point  is  to  use  the  right  tool  for  job. 

6.  Once  a  model  becomes  too  large,  spreadsheet  tools  become  difficult  to  manage. 
This  became  very  evident  during  model  development  and  analysis.  Due  to  this  difficulty,  it 
is  recommended  that  other  information  technology  tools  be  investigated  for  potential  use 
as  the  model  base  in  the  schematic  DSS  model  resulting  from  investigative  question  one. 
On  their  internet  website,  Compass  Modeling  Solutions  advertises  a  large-scale  linear 
programming/mixed  integer  programming  (LP/MIP)  SOLVER  from  Frontline  which  has 
the  capacity  to  “solve  problems  with  up  to  6,500  variables,  and  to  upgrade  the  Microsoft 
Excel  SOLVER  program  to  make  it  significantly  faster  and  more  robust.”  Compass 
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claims  the  capabilities  of  the  spreadsheet  programs  such  as  Microsoft  Excel  and  Borland’s 
Quattro  Pro,  which  are  sold  with  integrated  optimizers,  are  limited.  Compass  claims  the 
Microsoft  Excel  SOLVER  program  can  only  solve  linear  programs  with  less  than  200 
variables  and  100  constraints  (Compass,  1998). 

IBM  Manufacturing  Solutions  advertises  “IBM  Application  Development  tools  as 
the  basis  for  solving  many  types  of  linear  programming,  mixed  integer  programming,  as 
well  as  other  intelligent  heuristic  and  solver  techniques”  (IBM,  1998).  Two  specific  tools 
advertised  by  IBM  include  an  Optimization  Subroutine  Library  and  a  Mathematical 
Programming  Systems  Extended  (MPSX)  program.  The  Optimization  Subroutine  Library 
is  “a  suite  of  subroutines  for  manipulating  models  and  solving  the  resulting  minimization 
problems  of  mathematical  optimization”  (IBM,  1998).  The  library  includes  solvers  for 
linear  programming,  quadratic  programming,  and  mixed  integer  programming.  The 
MPSX  is  a  “decision  sciences  tool  that  is  used  to  make  better  business  decisions  [by  using] 
advanced  linear  and  mixed  integer  programming  to  analyze  complex  business 
environments  to  select  the  best  course  of  action  among  many  feasible  alternatives  (IBM, 
1998).  Many  software  programs  exist  to  implement  management  science  tools  efficiently 
and  effectively.  Prior  to  design  and  implementation  of  a  standard  evaluation  tool  in  the 
acquisition  field,  examination  of  the  various  software  programs  should  be  conducted  to 
ensure  consistent  accuracy  and  reliability  of  each  model. 

5.3  Conclusion 

In  conclusion,  the  results  of  the  research  proved  to  re-enforce  the  need  for  the 
Government  to  develop  information  technology  systems  using  more  strategic  decision- 
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making  methods,  such  as  found  in  the  field  of  management  science.  Expert  Systems  using 
artificial  intelligence.  Decision  Support  Systems  using  “if-then”  statements  or  case  based 
reasoning,  and  evaluation  models  implementing  integer  linear  programming  or  analytical 
hierarchy  processes  are  just  some  of  the  tools  provided  through  management  science 
which  can  enhance  the  decision-making  ability  of  the  SSA  in  determining  the  best 


alternative  to  the  Government. 


APPENDIX  A.  FAR  15-304 
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(a)  The  award  decision  is  based  on  evaluation  factors  and  significant 

subfactors  that  are  tailored  to  the  acquisition. 

(b)  Evaluation  factors  and  significant  subfactors  must-- 

(1)  Represent  the  key  areas  of  importance  and  emphasis  to  be  considered 
in  the  source  selection  decision;  and 

(2)  Support  meaningful  comparison  and  discrimination  between  and  among 
competing  proposals. 

(c)  The  evaluation  factors  and  significant  subfactors  that  apply  to  an 
acquisition  and  their  relative  importance  are  within  the  broad 
discretion  of  agency  acquisition  officials,  subject  to  the  following 
requirements: 

(1)  Price  or  cost  to  the  Government  shall  be  evaluated  in  every  source 
selection  (10  U.S.C.  2305(a)(3)(A)  (ii)  and  41  U.S.C.  253a(c)(l)(B)); 

(2)  The  quality  of  the  product  or  service  shall  be  addressed  in  every 
source  selection  through  consideration  of  one  or  more  non-cost 
evaluation  factors  such  as  past  performance,  compliance  with 
solicitation  requirements,  technical  excellence,  management  capability, 
personnel  qualifications,  and  prior  experience  (10  U.S.C.  2305(a)(3) 

(A)(i)  and  41  U.S.C.  253a(c)(l)(A));  and  [DoD  Deviation  97-00009] 

(3)(i)  Except  as  set  forth  in  paragraph  (c)(3)(iii)  of 
this  section,  past  performance  shall  be  evaluated 
in  all  source  selections  for  negotiated  competitive 
acquisitions  expected  to  exceed  $5,000,000  for 
Systems  and  Operations  Support  or  expected  to 
exceed  $1,000,000  for  all  other  acquisitions. 

(ii)  Past  performance  need  not  be  evaluated  if  the 
contracting  officer  documents  the  reason  past 
performance  is  not  an  appropriate  evaluation  factor 
for  the  acquisition  (OFPP  Policy  Letter  92-5).  [SIC] 

(3) (i)  Except  as  set  forth  in  paragraph  (c)(3)(iii)  of  this  section, 
past  performance  shall  be  evaluated  in  all  source  selections  for 
negotiated  competitive  acquisitions  expected  to  exceed  $1,000,000. 

(ii)  Except  as  set  forth  in  paragraph  (c)(3)(iii)  of  this  section,  past 
performance  shall  be  evaluated  in  all  source  selections  for  negotiated 
competitive  acquisitions  issued  on  or  after  January  1,  1999,  for 
acquisitions  expected  to  exceed  $100,000.  Agencies  should  develop 
phase-in  schedules  that  meet  or  exceed  this  schedule. 

(iii)  Past  performance  need  not  be  evaluated  if  the  contracting  officer 
documents  the  reason  past  performance  is  not  an  appropriate  evaluation 
factor  for  the  acquisition. 

(d)  All  factors  and  significant  subfactors  that  will  affect  contract  award 
and  their  relative  importance  shall  be  stated  clearly  in  the  solicitation  (10 
U.S.C.  2305(a)(2)(A)(i)  and  41  U.S.C.  253a(b)(l)(A)). 
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The  rating  method  need  not  be  disclosed  in  the  solicitation. 

The  general  approach  for  evaluating  past  performance  information 
shall  be  described. 

(e)  The  solicitation  shall  also  state,  at  a  minimum,  whether  all 
evaluation  factors  other  than  cost  or  price,  when  combined,  are- 

(1)  Significantly  more  important  than  cost  or  price; 

(2)  Approximately  equal  to  cost  or  price;  or 

(3)  Significantly  less  important  than  cost  or  price  (10  U.S.C. 
2305(a)(3)(A)(iii)  and  41  U.S.C.  253a(c)(l)(C)). 
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Appendix  B.  A  Contract  Award  Problem 
(Ragsdale,  1997:  234-241) 
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Sample  Problem-Integer  Linear  Programming 

B&G  Construction  is  a  commercial  building  company  located  in  Tampa,  Florida. 
The  Company  has  recently  signed  contracts  to  construct  four  buildings  in  different 
locations  throughout  southern  Florida.  Each  building  project  requires  large  amounts  of 
cement  to  be  delivered  to  the  building  sites.  At  B&G’s  request,  three  cement  companies 
have  submitted  bids  for  supplying  the  cement  for  these  jobs.  The  following  table 
summarizes  the  prices  the  three  companies  charge  per  delivered  ton  of  cement  and  the 
maximum  amount  of  cement  that  each  company  can  provide. 

Cost  Per  Delivered  Ton  of  Cement 


Project  1 

Project  2  Project  3  Project  4  Max.  Supply 

Company  1 

$120 

$115 

$130 

$125 

525 

Company  2 

$100 

$150 

$110 

$105 

450 

Company  3 

$140 

$95 

$145 

$165 

550 

Total  Tons  Needed 

450 

275 

300 

350 

For  example,  company  1  can  supply  a  maximum  of  525  tons  of  cement,  and  each 
ton  delivered  to  projects  1,  2,  3,  and  4  will  cost  $120,  $115,  $130,  and  $125,  respectively. 
The  costs  vary  primarily  because  of  the  different  distances  between  the  cement  plants  and 
the  construction  sites.  The  numbers  in  the  last  row  of  the  table  indicate  the  total  amount 
of  cement  (in  tons)  required  for  each  project. 

In  addition  to  the  maximum  supplies  listed,  each  cement  company  places  special 
conditions  on  its  bid.  Specifically,  company  1  indicated  that  it  will  not  supply  orders  of 
less  than  150  tons  for  any  of  the  construction  projects.  Company  2  indicated  that  it  can 
supply  more  than  200  tons  to  no  more  than  one  of  the  projects.  Company  3  indicated  that 
it  will  accept  only  orders  that  total  200  tons,  400  tons,  or  550  tons. 
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B&G  can  contract  with  no  more  than  one  supplier  to  meet  the  cement 
requirements  for  a  given  project.  The  problem  is  to  determine  what  amounts  to  purchase 
from  each  supplier  to  meet  the  demands  for  each  project  at  the  least  total  cost. 


1) .  To  begin  formulating  this  problem,  first  define  the  decision  variables  as: 

Xij  =  tons  of  cement  purchased  from  company  i  for  construction  project,/ 

2) .  Establish  the  objective  function  to  minimize  total  cost  as: 

MIN:  120Xn+  115Xi2+  130Xi3  +  125X14 
+  IOOX21  +  150X22  +  1 IOX23  +  105X24 
+  HOXsi  +  95X32  +  145X33  +  165X34 

3) .  Establish  the  constraints  to  ensure  the  maximum  supply  of  cement  from  each  company 

is  not  exceeded: 

Xu  +  X12  +  X13  +  Xw  <  525  }  supply  from  company  1 

X2i  +  X22  +  X23  +  X24  <  450  }  supply  from  company  2 

X31  +  X32  +  X33  +  X34  <  550  }  supply  from  company  3 

4) .  Establish  the  constraints  to  ensure  the  requirements  for  cement  at  each  construction 

project  are  met: 

Xn+  X21  +  X31  =  450  }  demand  for  cement  at  project  1 
X12  +  X22  +  X32  =  275  }  demand  for  cement  at  project  2 
X13  +  X23  +  X33  =  300  }  demand  for  cement  at  project  3 
X14  +  X24  +  X34  =  350  }  demand  for  cement  at  project  4 

5) .  Figure  6  demonstrates  how  the  objective  function  and  constraints  of  this  problems  are 
implemented  into  a  spreadsheet  model.  In  this  spreadsheet,  the  costs  per  delivered  ton  of 
cement  are  shown  in  cells  B3  through  E5.  Cells  B9  through  Ell  represent  the  decision 
variables  in  the  model.  The  objective  function  is  entered  in  cell  G13  as: 

Formula  for  cell  G13:  =SUMPRODUCT  (B3:E5,B9:E1 1) 

The  left-hand  side  (LHS)  formulas  of  the  supply  constraints  are  entered  in  cells  F9 
through  FI  1  as: 

Formula  for  cell  F9:  =SUM(B9:E9) 

(Copy  to  F 1 0  through  F 1 1 ) 
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Cells  G9  through  G1 1  contain  the  right-hand  side  (RHS)  values  for  these 
constraints.  The  LHS  formulas  for  the  demand  constraints  are  entered  in  cells  B 12 
through  E 12  as: 

Formula  for  cell  B12:  =SUM(B9:B1 1) 

(Copy  to  C12  through  El 2) 


Cells  B13  through  El 3  contain  the  RHS  values  for  these  constraints. 

6).  Formulate  the  side-constraints.  Company  1  indicated  that  it  will  not  accept  orders  for 
less  than  150  tons  for  any  of  the  construction  projects.  This  minimum  order-size 
restriction  is  modeled  by  the  following  eight  constraints,  where  Yy  represent  binary 
variables: 

Xn<525Y„ 

X,2<525Yi2 
X,3  <  525Yi3 
X14  <  525 Y14 
Xu>525Yn 
Xn  >525Y12 
X,3  >525 Yis 
Xi4  >525Yh 

Each  constraint  has  an  algebraically  equivalent  constraint,  which  will  ultimately  be 
used  in  implementing  the  constraint  in  the  spreadsheet.  The  first  four  constraints  represent 
linking  constraints  that  ensure  if  Xu,  Xi2,  Xi3,  orX14  is  greater  than  0,  then  its  associated 
binary  variable  (Yn,  Yi2,  Y!3,  or  Y!4)  must  be  equal  to  1 .  (These  constraints  also  indicate 
that  525  is  the  maximum  value  that  can  be  assumed  by  Xu,  X12,  X13,  and  X14).  The  next 
four  constraints  ensure  that  if  Xii,Xi2,X13,orXi4  is  greater  than  0,  it  must  be  at  least  150. 
These  constraints  are  included  in  the  model  to  ensure  that  any  order  given  to  company  1  is 
for  at  least  150  tons  of  cement. 

Company  2  indicated  that  it  can  supply  more  than  200  tons  to  no  more  than  one  of 
the  projects.  This  type  of  restriction  is  represented  by  the  following  set  of  constraints 
where  the  Yy  represent  binary  variables: 

X2i  <  200  +  250Y2i  (implement  as  X2i  -  200  -  250Y2i  <  0) 

X22  ^  200  +  250Y22  (implement  as  X22  -  200  -  250Y22  ^  0) 

X23  <  200  +  250Y23  (implement  as  X23  -  200  -  250Y23  ^  0) 

X24  <  200  +  25OY24  (implement  as  X24  -  200  -  250Y24  ^  0) 

Y21  +  Y22+Y23+Y24<  1  (implement  as  is) 
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The  first  constraint  indicates  that  the  amount  supplied  from  company  2  for  project 
1  must  be  less  than  200  if  Y2\  =  0,  or  less  than  450  (the  maximum  supply  form  company 
2)  if  Y21  =  1 .  The  next  three  constraints  have  similar  interpretations  for  the  amount 
supplied  from  company  2  to  projects  2,  3,  and  4,  respectively.  The  last  constraint 
indicates  that  at  most  one  of  Y2i,  Y22,  Y23,  and  Y24  can  equal  1.  Therefore,  only  one  of 
the  projects  can  receive  more  than  200  tons  of  cement  from  company  2. 

The  final  set  of  constraints  for  this  problem  addresses  company  3’s  stipulation  that 
it  will  accept  only  orders  totaling  200,  400,  or  550  tons.  This  type  of  condition  is  modeled 
using  binary  Yy  variables  as: 


X31  +  X32  +  X33+  X34=  200 Y3,  +  400Y32  +  550Y33 

(implement  as  X3i  +  X32+  X33  +  X34  -  200Y3i  -  400Y32  -  550Y33  =  0) 

Y3i+Y32+  Y33  <  1  (implement  as  is) 

These  constraints  allow  for  the  total  amount  ordered  from  company  3  to  assume 
four  distinct  values.  If  Y3i  =  Y32=  Y33  =  0,  then  no  cement  will  be  ordered  from  company 
3.  If  Y3i  =  1,  then  200  tons  must  be  ordered.  If  Y32  =  1,  then  400  tons  must  be  ordered. 
Finally,  if  Y33  =  1,  then  550  tons  must  be  ordered  from  company  3.  These  two  constraints 
enforce  the  special  condition  imposed  by  company  3. 

7).  Figure  18  demonstrates  how  the  side-constraints  are  implemented  into  a  spreadsheet 
model.  The  side-constraints  in  this  problem  impose  restrictions  on  the  feasible  solutions 
that  can  be  considered,  but  these  constraints  serve  a  more  “mechanical”  purpose — to  make 
the  model  work — and  are  not  of  primary  interest  to  management.  As  such,  it  is  often 
convenient  to  implement  side-constraints  in  an  out-of-the-way  area  of  the  spreadsheet  so 
they  do  not  detract  from  the  primary  purpose  of  the  spreadsheet. 

To  implement  the  side-constraints  for  company  1,  we  enter  a  batch-size  restriction 
of  150  in  cell  B17  and  reserve  cells  B18  through  E18  to  represent  the  binary  variables  Yu, 
Yi2.  Yi3.  and  YK.  The  LHS  formulas  for  the  linking  constraints  for  company  1  are 
implemented  in  cells  B19  through  E19  as: 

Formula  for  cell  B 19:  -B9-$G$9*B18 

(Copy  to  C19  through  E19) 


Cell  FI 9  contains  a  reminder  to  tell  SOLVER  that  these  cells  must  be  less  than  or 
equal  to  0.  The  LHS  formulas  for  the  batch-size  constraints  for  company  1  are 
implemented  in  cells  B20  through  E20  as: 

Formula  for  cell  B20:  =B9-$B$  1 7*B  1 8 

(Copy  to  C20  through  E20) 
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Cell  F20  contains  a  reminder  to  tell  SOLVER  that  these  cells  must  be  greater  than 
or  equal  to  0. 

To  implement  the  side-constraints  for  company  2,  the  maximum  supply  value  of 
200  is  entered  in  ceil  B22  and  reserve  cells  B23  through  E23  to  represent  binary  variables 
Y21.  Y22.  Y23.  and  Y24.  The  LHS  formulas  for  the  maximum  supply  constraints  are 
implemented  in  cells  B24  through  E24  as: 

Formula  for  cell  B24:  =B  1 0-$B$22-($G$  1 0-$B$22)*B23 

(Copy  to  C24  tlirough  E24) 


Cell  F24  contains  a  reminder  to  tell  SOLVER  that  these  cells  must  be  less  than  or 
equal  to  0.  To  ensure  that  no  more  than  one  order  from  company  2  exceeds  200  tons,  the 
sum  of  the  binary  variable  for  company  2  cannot  exceed  1 .  The  LHS  formula  for  this 
constraint  is  entered  in  cell  E25  as: 

Formula  for  cell  E25:  =SUM(B23 :E23) 

Cell  F25  contains  a  reminder  to  tell  SOLVER  that  this  cell  must  be  less  than  or 
equal  to  1 . 

To  implement  the  side-constraints  for  company  3,  the  three  possible  total  order 
amounts  are  entered  in  cells  B27  through  D27.  Cells  B28  through  D28  are  reserved  to 
represent  the  binary  variables  Y3i,  Y32,  and  Y33.  The  LHS  formula  for  company  3’s  total 
supply  side-constraint  is  entered  in  cell  D29  as: 

Formula  for  cell  D29:  =SUM(B1 1  :E1 1  )-SUMPRODUCT(B27 :D27,B28 :D28) 

Cell  E29  contains  a  reminder  to  tell  SOLVER  that  cell  D29  must  equal  0.  To 
ensure  that  no  more  than  one  of  the  binary  variables  for  company  3  is  set  equal  to  1,  the 
sum  of  these  variables  is  entered  in  cell  D30  as: 

Formula  for  cell  D30:  =SUM(B28:D28) 

Cell  E3  0  contains  a  reminder  to  tell  SOLVER  that  this  cell  must  be  less  than  or 
equal  to  1 
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A 

B 

c 

D 

E 

F 

G 

1 

Cost  Per  Delivered  Ton 

2 

Project  1  Project  2 

Project  3  Project  4 

3  Company  1 

$120 

$115 

$130 

$125 

4  Company  2 

$100 

$150 

$110 

$105 

5  Company  3 

$140 

$95 

$145 

$165 

6 

7 

Amount  to  Purchase 

8 

Project  1  Project  2  Project  3  Project  4 

Supplied 

Available 

9  Company  1 

0 

0 

0 

0 

0 

525 

10  Company  2 

0 

0 

0 

0 

0 

450 

1 1  Company  3 

0 

0 

0 

0 

0 

550 

12  Received 

0 

0 

0 

0 

13  Needed 

450 

275 

300 

350 

14 

Total 

$0 

Cost: 

1 5  Additional  Constraints  Follow: 

16 

1 7  Co.  1  Batch-Size 

150 

18  Binary  Variables 

0 

0 

0 

0 

19  Linking 

0 

0 

0 

0 

(<=0) 

Constraints 

20  Batch  Constraints 

0 

0 

0 

0 

(>=  0) 

21 

22  Co.  2  Max  Supply 

200 

23  Binary  Variables 

0 

0 

0 

0 

24  UB  Constraints 

-200 

-200 

-200 

-200 

(<=0) 

25  Sum  of  Binary  Variables 

0 

(<=1) 

26 

27  Co.  3  Tot.  Supply 

200 

400 

550 

28  Binary  Variables 

0 

0 

0 

29  Tot.  Supply  =  0,  200 

,400, 

0.00 

(=0) 

550 

30  Sum  of  Binary  Variables 

0 

Figure  18.  Objective  Function  and  Constraints  Implemented  in  a  Spreadsheet 


Model. 
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8).  Solving  the  model.  The  SOLVER  parameters  and  options  required  to  complete  this 
program  are  as  follows: 


SOLVER  PARAMETERS 

G14 
Min 

B9:E11,  B18:E18,  B23:E23,  B28:D28 

B9:E11  >0 
B18:E18  >  0 
B18:E18  <  1 
B18:E18  =  integer 
B23:E23  >0 
B23  :E23  <  1 
B23  :E23  =  integer 
B28:E28  >  0 
B28:E28  <  1 
B28:E28  =  integer 
F9:F11  <  G9:G11 
B12:E12  =  B13:E13 
B19:E19  <  0 
B20:E20  >  0 
B24:E24  <  0 
E25  <  1 
D29  =  0 
D30  <  1 

SOLVER  OPTIONS 

Assume  Linear  Model  Tolerance  =  0% 

Use  Automatic  Scaling  Iterations  =100 

9).  An  optimal  solution  is  demonstrated  in  Figure  19.  (There  are  alternative  optimal 
solutions  to  this  problem.)  In  this  solution,  the  amounts  of  cement  required  by  each 
construction  project  are  met  exactly.  Aso,  each  condition  imposed  by  the  side-constraints 
for  each  company  is  met.  Specifically,  the  orders  awarded  to  company  1  are  tor  at  least 
150  tons;  only  one  of  the  orders  awarded  to  company  2  exceeds  200  tons;  and  the  sum  of 
the  orders  awarded  to  company  3  is  exactly  equal  to  400  tons. 


Set  Target  Cell: 

Equal  to: 

By  Changing  Cells: 

Subject  to  the  Constraints: 
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A 

B 

c 

D 

E 

F 

G 

1 

Cost  Per  Delivered  Ton 

2 

Project  1  Project  2  Project  3  Project  4 

3  Company  1 

$120 

$115 

$130 

$125 

4  Company  2 

$100 

$150 

$110 

$105 

5  Company  3 

$140 

$95 

$145 

$165 

6 

7 

Amount  to  Purchase 

8 

Project  1  Project  2  Project  3  Project  4 

Supplied 

Available 

9  Company  1 

175 

0 

0 

350 

525 

525 

1 0  Company  2 

275 

0 

175 

0 

450 

450 

1 1  Company  3 

0 

275 

125 

0 

400 

550 

12  Received 

450 

275 

300 

350 

13  Needed 

450 

275 

300 

350 

14 

Total 

$155,750 

Cost: 

15  Additional  Constraints  Follow: 

16 

17  Co.  1  Batch-Size 

150 

18  Binary  Variables 

1 

0 

0 

1 

19  Linking 

-350 

0 

3.23E-10 

-175 

(<=0) 

Constraints 

20  Batch  Constraints 

25 

0 

3.23E-10 

200 

(>=0) 

21 

22  Co.  2  Max  Supply 

200 

23  Binary  Variables 

1 

0 

0 

0 

24  UB  Constraints 

-175 

-200 

-25 

-200 

(<=0) 

25  Sum  of  Binary  Variables 

1 

(<=1) 

26 

27  Co.  3  Tot.  Supply 

200 

400 

550 

28  Binary  Variables 

0 

1 

0 

29  Tot.  Supply  =  0,  200,  400,  550 

0.00 

(=0) 

30  Sum  of  Binary  Variables 

1 

(<=i) 

Figure  19.  Optimal  Solution  to  B&G’s  Contract  Award  Problem. 


10).  A  Microsoft  Excel  Answer  Summary  Report  is  provided  on  the  next  four  pages. 

This  summary  report  synopsizes  the  final  values  for  the  target  and  adjustable  cells.  In 
addition,  it  indicates  which  constraints  are  binding  or  non-binding  on  the  final  solution  and 
also  reports  how  much  slack  is  left  in  the  variable,  or  how  much  the  final  value  for  that 
constraint  can  change  before  it  affects  outcome  of  the  final  solution.  This  report  is 
automatically  generated  and  can  be  provided  to  management  as  a  summary  of  the 
problem’s  solution. 
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(Continued  from  Previous  Page) 

$C$23  B inary  Variabl es  Proj ect  2 
$D$2 3  JB i nary  V ari ab les  Project  3 
$E$23  Binary  Variables  Project  4 
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Appendix  C.  Sealed  Bidding  SOLVER  Model 
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Figure  22.  Basic  Sealed  Bid  Model  Layout  (Aggregate  Award  -  Part  3). 
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Figure  24.  Basic  Sealed  Bid  Model  Layout  (Multiple  Awards). 
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Table  8 


Solver  Parameters  and  Options 
(Sealed  Bid  -  Aggregate  and  Multiple  Awards). 

PARAMETERS  (Sealed  Bid  -  Aggregate  Award) 

Set  Target  Cell:  J21 

Equal  to:  Min 

By  Changing  Cells:  D  17:120,  D26:I29,  D32:I32 

Subject  to  the  Constraints: 

D1 7:120  =  INT 
D17:I20  >  0 
D26:I29  =  BIN 
D26:I29  =  INT 
D32:I32  <  D99:I99 
D32.I32  =  BIN 
D32:I32  =  INT 
D36:I39  <  0 
D4 1:141  <0 
J17:J20  >K17:K20 
J26:J29  =  1 
J32  <  1 


PARAMETERS  (Sealed  Bid  -  Multiple  Awards) 


Set  Target  Cell: 

Equal  to: 

By  Changing  Cells: 

Subject  to  the  Constraints: 


J21 

Min 

D  17:120 

D17:I20  =  INT 
D1 7:120  >  0 
D25:I25  <D85:I85 
J17:J20  >K17:K20 


SOLVER  OPTIONS  FOR  BOTH  MODELS 

Assume  Linear  Model  Tolerance  =  0% 

Use  Automatic  Scaling  Iterations  =  1000 
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Appendix  D.  FAR  Part  15,  Competitive  Proposal  SOLVER  Model 
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Frequency  Conversion  of  Ratings 
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Figure  26.  Basic  FAR  Part  15  Model  Layout  (Technical  Criteria  Rating  Charts). 
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Figure  27.  Technical  Criteria  Ratings. 
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Table  9 


Solver  Parameters  and  Options 
(FAR  Part  15-  Aggregate  Award). 


PARAMETERS 

Set  Target  Cell: 

Equal  to: 

By  Changing  Cells: 

Subject  to  the  Constraints: 


122 

Min 

D1 8:121,  D27:I30,  D33:I33 

D1 8:121  =  INT 
D18:I21  >0 
D27:I30  =  BIN 
D27:I30  =  INT 
D33:I33  <D109:I109 
D33:I33  =  BIN 
D33:I33  =  INT 
D37:I40  <  0 
D42:I42  <  0 
J18:J21  >K18:K21 
J27:J30  =  1 
J33  <  1 


SOLVER  OPTIONS 

Assume  Linear  Model  Tolerance  =  0% 

Use  Automatic  Scaling  Iterations  =  1000 
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